{ "cells": [ { "cell_type": "markdown", "metadata": { "id": "fwukZZnNTYWE" }, "source": [ "
\n", "\n", "# Data Acquisition and Preprocessing\n", "\n", "Copyright, NLP from scratch, 2024.\n", "\n", "[NLPfor.me](https://www.nlpfor.me)\n", "\n", "------------" ] }, { "cell_type": "markdown", "metadata": { "id": "wG0CakglfdmV" }, "source": [ "## Data Acqusition\n", "\n", "### Requesting Data From a Web Service with the `requests` library" ] }, { "cell_type": "markdown", "metadata": { "id": "vW18g_6ofdmX" }, "source": [ "In this notebook, we will acquire and preprocess text data from online sources. We have already been introduced to the [requests library](https://requests.readthedocs.io/en/latest/) and we will show how using it, with a few simple lines of code, we can pull data from a web service (REST API).\n", "\n", "[The Cocktail DB](https://www.thecocktaildb.com/) is an open source database of cocktails and drinks from around the world, and their ingredients. It also has an API that is free to use for educational purposes.\n", "\n", "Let's get some text data using the `requests` library, here a description of gin. The URL pattern for a given web service is up to its designer, and should be well documented. The Cocktail DB tells us to use the URL pattern `https://www.thecocktaildb.com/api/json/v1/1/search.php?i=` in order to get information back on a drink ingredient.\n", "\n", "First, we import the requests library, than simply make a request using the `get` method and the URL:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "tdOoclo3xX1f" }, "outputs": [], "source": [ "# Import the requests library\n", "import requests\n", "\n", "# Make a call to the API\n", "r = requests.get(\"https://www.thecocktaildb.com/api/json/v1/1/search.php?i=gin\")" ] }, { "cell_type": "markdown", "metadata": { "id": "fch1ViDGfdmb" }, "source": [ "Our machine has now made the request and hopefully gotten a response from the server! Let's check the response code." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "executionInfo": { "elapsed": 29, "status": "ok", "timestamp": 1687981819911, "user": { "displayName": "Myles Harrison", "userId": "02225822412878142066" }, "user_tz": 240 }, "id": "v3j4XGvdxhr1", "outputId": "811a3ebd-ee84-4a7e-ca32-d6d397184f0b" }, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Check the response\n", "r" ] }, { "cell_type": "markdown", "metadata": { "id": "MkJ7g3Fifdme" }, "source": [ "We can see we have received a response code of 200, which means \"OK\" and that data was returned successfully. Let's check what was returned from our request. There are two ways to do this: the most straightforward is just to return using the `.text` attribute, which shows the contents of the response as an ordinary python string:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 123 }, "executionInfo": { "elapsed": 25, "status": "ok", "timestamp": 1687981819912, "user": { "displayName": "Myles Harrison", "userId": "02225822412878142066" }, "user_tz": 240 }, "id": "Zvs8PgFuxix5", "outputId": "63bd2faa-b121-404d-fa63-46e8779922a3" }, "outputs": [ { "data": { "application/vnd.google.colaboratory.intrinsic+json": { "type": "string" }, "text/plain": [ "'{\"ingredients\":[{\"idIngredient\":\"2\",\"strIngredient\":\"Gin\",\"strDescription\":\"Gin is a distilled alcoholic drink that derives its predominant flavour from juniper berries (Juniperus communis). Gin is one of the broadest categories of spirits, all of various origins, styles, and flavour profiles, that revolve around juniper as a common ingredient.\\\\r\\\\n\\\\r\\\\nFrom its earliest origins in the Middle Ages, the drink has evolved from a herbal medicine to an object of commerce in the spirits industry. Gin emerged in England after the introduction of the jenever, a Dutch liquor which originally had been a medicine. Although this development had been taking place since early 17th century, gin became widespread after the William of Orange-led 1688 Glorious Revolution and subsequent import restrictions on French brandy.\\\\r\\\\n\\\\r\\\\nGin today is produced in subtly different ways, from a wide range of herbal ingredients, giving rise to a number of distinct styles and brands. After juniper, gin tends to be flavoured with botanical\\\\/herbal, spice, floral or fruit-flavours or often a combination. It is most commonly consumed mixed with tonic water. Gin is also often used as a base spirit to produce flavoured gin-based liqueurs such as, for example, Sloe gin, traditionally by the addition of fruit, flavourings and sugar.\",\"strType\":\"Gin\",\"strAlcohol\":\"Yes\",\"strABV\":\"40\"}]}'" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# What are the contents\n", "r.text" ] }, { "cell_type": "markdown", "metadata": { "id": "LBzUfL-Mfdmh" }, "source": [ "We can see there is some nesting of data here, as the response is actually returned in [Javascript Object Notation (JSON) format ](https://en.wikipedia.org/wiki/JSON), or what someone who works in python might instead call a [dictionary](https://docs.python.org/3/tutorial/datastructures.html#dictionaries). We can see the resposne in JSON format as well using `.json` method of the response object:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "executionInfo": { "elapsed": 22, "status": "ok", "timestamp": 1687981819913, "user": { "displayName": "Myles Harrison", "userId": "02225822412878142066" }, "user_tz": 240 }, "id": "6LamGPIxxlFi", "outputId": "01ef4ec3-ccd8-46f6-e7c3-da0ecf42de57" }, "outputs": [ { "data": { "text/plain": [ "{'ingredients': [{'idIngredient': '2',\n", " 'strIngredient': 'Gin',\n", " 'strDescription': 'Gin is a distilled alcoholic drink that derives its predominant flavour from juniper berries (Juniperus communis). Gin is one of the broadest categories of spirits, all of various origins, styles, and flavour profiles, that revolve around juniper as a common ingredient.\\r\\n\\r\\nFrom its earliest origins in the Middle Ages, the drink has evolved from a herbal medicine to an object of commerce in the spirits industry. Gin emerged in England after the introduction of the jenever, a Dutch liquor which originally had been a medicine. Although this development had been taking place since early 17th century, gin became widespread after the William of Orange-led 1688 Glorious Revolution and subsequent import restrictions on French brandy.\\r\\n\\r\\nGin today is produced in subtly different ways, from a wide range of herbal ingredients, giving rise to a number of distinct styles and brands. After juniper, gin tends to be flavoured with botanical/herbal, spice, floral or fruit-flavours or often a combination. It is most commonly consumed mixed with tonic water. Gin is also often used as a base spirit to produce flavoured gin-based liqueurs such as, for example, Sloe gin, traditionally by the addition of fruit, flavourings and sugar.',\n", " 'strType': 'Gin',\n", " 'strAlcohol': 'Yes',\n", " 'strABV': '40'}]}" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Convert to JSON and dict\n", "r.json()" ] }, { "cell_type": "markdown", "metadata": { "id": "9dwqvXPrfdmj" }, "source": [ "Now to pull out the description, it is a matter of subsetting the returned list associated with the `ingredients` key (there is only one element, element 0) and the getting the value associated with the `strDescription` key within it:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "executionInfo": { "elapsed": 19, "status": "ok", "timestamp": 1687981819914, "user": { "displayName": "Myles Harrison", "userId": "02225822412878142066" }, "user_tz": 240 }, "id": "R7r7BuVifdmj", "outputId": "5232e4c7-ffa4-413d-bd03-e3013f44112f" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Gin is a distilled alcoholic drink that derives its predominant flavour from juniper berries (Juniperus communis). Gin is one of the broadest categories of spirits, all of various origins, styles, and flavour profiles, that revolve around juniper as a common ingredient.\r\n", "\r\n", "From its earliest origins in the Middle Ages, the drink has evolved from a herbal medicine to an object of commerce in the spirits industry. Gin emerged in England after the introduction of the jenever, a Dutch liquor which originally had been a medicine. Although this development had been taking place since early 17th century, gin became widespread after the William of Orange-led 1688 Glorious Revolution and subsequent import restrictions on French brandy.\r\n", "\r\n", "Gin today is produced in subtly different ways, from a wide range of herbal ingredients, giving rise to a number of distinct styles and brands. After juniper, gin tends to be flavoured with botanical/herbal, spice, floral or fruit-flavours or often a combination. It is most commonly consumed mixed with tonic water. Gin is also often used as a base spirit to produce flavoured gin-based liqueurs such as, for example, Sloe gin, traditionally by the addition of fruit, flavourings and sugar.\n" ] } ], "source": [ "description = r.json()['ingredients'][0]['strDescription']\n", "print(description)" ] }, { "cell_type": "markdown", "metadata": { "id": "paMiILuufdmk" }, "source": [ "Great! We have sucessfully retrieved some text from an API using `requests`. We could write more code to return more data programatically and stored in a data structure such as a list or pandas dataframe to work with in an NLP task:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "sJvcDKHefdmk" }, "outputs": [], "source": [ "# List of ingredients for building request URLs\n", "ingredients = ['gin', 'vodka', 'rum']\n", "\n", "# Empty list to hold descriptions returned from API\n", "description_list = list()\n", "\n", "# Iterate over the ingredients\n", "for ingredient in ingredients:\n", "\n", " # Make a call to the API\n", " r = requests.get(f\"https://www.thecocktaildb.com/api/json/v1/1/search.php?i={ingredient}\")\n", "\n", " # Pull out the description and append to the list\n", " description = r.json()['ingredients'][0]['strDescription']\n", " description_list.append({'ingredient':ingredient, 'description':description})" ] }, { "cell_type": "markdown", "metadata": { "id": "dJky8PQyfdml" }, "source": [ "Now we have a list storing the description field from data returned from the API calls:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "executionInfo": { "elapsed": 10, "status": "ok", "timestamp": 1687981820144, "user": { "displayName": "Myles Harrison", "userId": "02225822412878142066" }, "user_tz": 240 }, "id": "uSOPLzQ9fdml", "outputId": "036023a7-e089-4ee0-a258-03ff6b489aa9", "scrolled": true }, "outputs": [ { "data": { "text/plain": [ "[{'ingredient': 'gin',\n", " 'description': 'Gin is a distilled alcoholic drink that derives its predominant flavour from juniper berries (Juniperus communis). Gin is one of the broadest categories of spirits, all of various origins, styles, and flavour profiles, that revolve around juniper as a common ingredient.\\r\\n\\r\\nFrom its earliest origins in the Middle Ages, the drink has evolved from a herbal medicine to an object of commerce in the spirits industry. Gin emerged in England after the introduction of the jenever, a Dutch liquor which originally had been a medicine. Although this development had been taking place since early 17th century, gin became widespread after the William of Orange-led 1688 Glorious Revolution and subsequent import restrictions on French brandy.\\r\\n\\r\\nGin today is produced in subtly different ways, from a wide range of herbal ingredients, giving rise to a number of distinct styles and brands. After juniper, gin tends to be flavoured with botanical/herbal, spice, floral or fruit-flavours or often a combination. It is most commonly consumed mixed with tonic water. Gin is also often used as a base spirit to produce flavoured gin-based liqueurs such as, for example, Sloe gin, traditionally by the addition of fruit, flavourings and sugar.'},\n", " {'ingredient': 'vodka',\n", " 'description': 'Vodka is a distilled beverage composed primarily of water and ethanol, sometimes with traces of impurities and flavorings. Traditionally, vodka is made by the distillation of fermented cereal grains or potatoes, though some modern brands use other substances, such as fruits or sugar.\\r\\n\\r\\nSince the 1890s, the standard Polish, Russian, Belarusian, Ukrainian, Estonian, Latvian, Lithuanian and Czech vodkas are 40% alcohol by volume ABV (80 US proof), a percentage that is widely misattributed to Dmitri Mendeleev. The European Union has established a minimum of 37.5% ABV for any \"European vodka\" to be named as such. Products sold as \"vodka\" in the United States must have a minimum alcohol content of 40%. Even with these loose restrictions, most vodka sold contains 40% ABV. For homemade vodkas and distilled beverages referred to as \"moonshine\", see moonshine by country.\\r\\n\\r\\nVodka is traditionally drunk neat (not mixed with any water, ice, or other mixer), though it is often served chilled in the vodka belt countries (Belarus, Estonia, Finland, Iceland, Latvia, Lithuania, Norway, Poland, Russia, Sweden, Ukraine). It is also commonly used in cocktails and mixed drinks, such as the vodka martini, Cosmopolitan, vodka tonic, Screwdriver, Greyhound, Black or White Russian, Moscow Mule, and Bloody Mary.\\r\\n\\r\\nScholars debate the beginnings of vodka. It is a contentious issue because very little historical material is available. For many centuries, beverages differed significantly compared to the vodka of today, as the spirit at that time had a different flavor, color and smell, and was originally used as medicine. It contained little alcohol, an estimated maximum of about 14%, as only this amount can be attained by natural fermentation. The still, allowing for distillation (\"burning of wine\"), increased purity, and increased alcohol content, was invented in the 8th century.\\r\\n\\r\\nA common property of the vodkas produced in the United States and Europe is the extensive use of filtration prior to any additional processing including the addition of flavorants. Filtering is sometimes done in the still during distillation, as well as afterwards, where the distilled vodka is filtered through activated charcoal and other media to absorb trace amounts of substances that alter or impart off-flavors to the vodka. However, this is not the case in the traditional vodka-producing nations, so many distillers from these countries prefer to use very accurate distillation but minimal filtering, thus preserving the unique flavors and characteristics of their products.\\r\\n\\r\\nThe master distiller is in charge of distilling the vodka and directing its filtration, which includes the removal of the \"fore-shots\", \"heads\" and \"tails\". These components of the distillate contain flavor compounds such as ethyl acetate and ethyl lactate (heads) as well as the fusel oils (tails) that impact the usually desired clean taste of vodka. Through numerous rounds of distillation, or the use of a fractioning still, the taste is modified and clarity is increased. In contrast, distillery process for liquors such as whiskey, rum, and baijiu allow portions of the \"heads\" and \"tails\" to remain, giving them their unique flavors.\\r\\n\\r\\nRepeated distillation of vodka will make its ethanol level much higher than is acceptable to most end users, whether legislation determines strength limits or not. Depending on the distillation method and the technique of the stillmaster, the final filtered and distilled vodka may have as much as 95–96% ethanol. As such, most vodka is diluted with water prior to bottling.\\r\\n\\r\\nPolish distilleries make a very pure (96%, 192 proof, formerly also 98%) rectified spirit (Polish language: spirytus rektyfikowany). Technically a form of vodka, it is sold in liquor stores rather than pharmacies. Similarly, the German market often carries German, Hungarian, Polish, and Ukrainian-made varieties of vodka of 90 to 95% ABV. A Bulgarian vodka, Balkan 176°, has an 88% alcohol content. Everclear, an American brand, is also sold at 95% ABV.'},\n", " {'ingredient': 'rum',\n", " 'description': 'Rum is a distilled alcoholic beverage made from sugarcane byproducts, such as molasses, or directly from sugarcane juice, by a process of fermentation and distillation. The distillate, a clear liquid, is then usually aged in oak barrels.\\r\\n\\r\\nThe majority of the world\\'s rum production occurs in the Caribbean and Latin America. Rum is also produced in Scotland, Austria, Spain, Australia, New Zealand, Fiji, the Philippines, India, Reunion Island, Mauritius, South Africa, Taiwan, Thailand, Japan, the United States, and Canada.\\r\\n\\r\\nRums are produced in various grades. Light rums are commonly used in cocktails, whereas \"golden\" and \"dark\" rums were typically consumed straight or neat, on the rocks, or used for cooking, but are now commonly consumed with mixers. Premium rums are also available, made to be consumed either straight or iced.\\r\\n\\r\\nRum plays a part in the culture of most islands of the West Indies as well as in The Maritimes and Newfoundland. This beverage has famous associations with the Royal Navy (where it was mixed with water or beer to make grog) and piracy (where it was consumed as bumbo). Rum has also served as a popular medium of economic exchange, used to help fund enterprises such as slavery (see Triangular trade), organized crime, and military insurgencies (e.g., the American Revolution and Australia\\'s Rum Rebellion).\\r\\n\\r\\nThe precursors to rum date back to antiquity. Development of fermented drinks produced from sugarcane juice is believed to have first occurred either in ancient India or in China, and to have spread from there. An example of such an early drink is brum. Produced by the Malay people, brum dates back thousands of years. Marco Polo also recorded a 14th-century account of a \"very good wine of sugar\" that was offered to him in the area that became modern-day Iran.\\r\\n\\r\\nThe first distillation of rum took place on the sugarcane plantations of the Caribbean in the 17th century. Plantation slaves first discovered molasses, a byproduct of the sugar refining process, could be fermented into alcohol. Later, distillation of these alcoholic byproducts concentrated the alcohol and removed impurities, producing the first true rums. Tradition suggests rum first originated on the island of Barbados. However, in the decade of the 1620s, rum production was recorded in Brazil. A liquid identified as rum has been found in a tin bottle found on the Swedish warship Vasa, which sank in 1628.\\r\\n\\r\\nA 1651 document from Barbados stated, \"The chief fuddling they make in the island is Rumbullion, alias Kill-Divil, and thi is made of sugar canes distilled, a hot, hellish, and terrible liquor.\"'}]" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "description_list" ] }, { "cell_type": "markdown", "metadata": { "id": "7y7oc6Kkfdmm" }, "source": [ "Finally, we can plunk this into a pandas dataframe to make things a bit nicer:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 143 }, "executionInfo": { "elapsed": 596, "status": "ok", "timestamp": 1687981820734, "user": { "displayName": "Myles Harrison", "userId": "02225822412878142066" }, "user_tz": 240 }, "id": "OBGKY6K7fdmm", "outputId": "e26c9553-5409-4825-f279-f76b10164743" }, "outputs": [ { "data": { "text/html": [ "\n", "
\n", "
\n", "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
ingredientdescription
0ginGin is a distilled alcoholic drink that derive...
1vodkaVodka is a distilled beverage composed primari...
2rumRum is a distilled alcoholic beverage made fro...
\n", "
\n", " \n", " \n", " \n", "\n", " \n", "
\n", "
\n", " " ], "text/plain": [ " ingredient description\n", "0 gin Gin is a distilled alcoholic drink that derive...\n", "1 vodka Vodka is a distilled beverage composed primari...\n", "2 rum Rum is a distilled alcoholic beverage made fro..." ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Check ouput\n", "import pandas as pd\n", "\n", "# Create a pandas dataframe from the list of key-value pairs\n", "# Keys are the column name, values are the values for each row\n", "desc_df = pd.DataFrame(description_list)\n", "\n", "# Check\n", "desc_df.head()" ] }, { "cell_type": "markdown", "metadata": { "id": "GmL7RKb1fdmn" }, "source": [ "## Scraping Web Pages with BeautifulSoup\n", "\n", "Unfortunately, the data we need is not always available for download or from a web service. In some cases we may have to [web scrape](https://en.wikipedia.org/wiki/Web_scraping) data if it is \"locked up\" in pages that are meant to be viewed in the browser.\n", "\n", "It should be noted that web scraping is a bit of a gray area legally, but as with free and open APIs, be respectful of whomever is hosting the content and associated resources (*i.e.* do not make excessive requests, or scrape entire sites without permission).\n", "\n", "Because the data is locked up in the code of the web page (a \"beautiful soup\" of HTML, Javascript, CSS, and other languages) we may also have to apply some elbow grease and do some work to pull out the elements in the page code that we want. Fortunately, [BeautifulSoup](https://www.crummy.com/software/BeautifulSoup/bs4/doc/) makes this easy, and we'll see you can almost treat the page code like a searchable database.\n", "\n", "Here we will scrape some text from the excellent online resource [Interpretable Machine Learning](https://christophm.github.io/interpretable-ml-book/ ), in particular [this page](https://christophm.github.io/interpretable-ml-book/what-is-machine-learning.html). Given its simple page structure, this should be relatively straightforward to do. `requests` has already been imported above, and we will now import the `BeautifulSoup` class for use shortly (the library name in python for BeautifulSoup is `bs4`, as it is version 4):" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "L_oyARKmfdmn" }, "outputs": [], "source": [ "from bs4 import BeautifulSoup\n", "\n", "# Page to scrape\n", "url = 'https://christophm.github.io/interpretable-ml-book/what-is-machine-learning.html'" ] }, { "cell_type": "markdown", "metadata": { "id": "Gx4QKLt5fdmo" }, "source": [ "Next, we make a request for the page using `requests`, just like we did for an API. The difference here is we are hitting a web server, which will return a web page, normally requested by and rendered in a browser:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "OzMCZBq4fdmo" }, "outputs": [], "source": [ "# Make the request\n", "r = requests.get(url)" ] }, { "cell_type": "markdown", "metadata": { "id": "zy9HaNhLfdmp" }, "source": [ "Let's take a look at the first 2000 characters of the result as a string:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 123 }, "executionInfo": { "elapsed": 19, "status": "ok", "timestamp": 1687981820986, "user": { "displayName": "Myles Harrison", "userId": "02225822412878142066" }, "user_tz": 240 }, "id": "RMRj38oxfdmp", "outputId": "4c898de0-dc99-4300-d33c-d39c4fab5ebd" }, "outputs": [ { "data": { "application/vnd.google.colaboratory.intrinsic+json": { "type": "string" }, "text/plain": [ "'\\n\\n\\n\\n \\n \\n 2.2 What Is Machine Learning? | Interpretable Machine Learning\\n \\n \\n\\n \\n \\n \\n \\n \\n\\n \\n \\n \\n \\n \\n\\n\\n\\n\\n\\n\\n \\n \\n \\n \\n \\n\\n\\n\\n\\n\\n\n", "\n", "\n", "\n", "\n", "2.2 What Is Machine Learning? | Interpretable Machine Learning\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
\n", "
\n", "\n", "
\n", "
\n", "
\n", "
\n", "

\n", "Interpretable Machine Learning\n", "

\n", "
\n", "
\n", "
\n", "
\n", " Buy Book \n", "Buy\n", "
\n", "

2.2 What Is Machine Learning?

\n", "

Machine learning is a set of methods that computers use to make and improve predictions or behaviors based on data.

\n", "

For example, to predict the value of a house, the computer would learn patterns from past house sales.\n", "The book focuses on supervised machine learning, which covers all prediction problems where we have a dataset for which we already know the outcome of interest (e.g. past house prices) and want to learn to predict the outcome for new data.\n", "Excluded from supervised learning are for example clustering tasks (= unsupervised learning) where we do not have a specific outcome of interest, but want to find clusters of data points.\n", "Also excluded are things like reinforcement learning, where an agent learns to optimize a certain reward by acting in an environment (e.g. a computer playing Tetris).\n", "The goal of supervised learning is to learn a predictive model that maps features of the data (e.g. house size, location, floor type, …) to an output (e.g. house price).\n", "If the output is categorical, the task is called classification, and if it is numerical, it is called regression.\n", "The machine learning algorithm learns a model by estimating parameters (like weights) or learning structures (like trees).\n", "The algorithm is guided by a score or loss function that is minimized.\n", "In the house value example, the machine minimizes the difference between the estimated house price and the predicted price.\n", "A fully trained machine learning model can then be used to make predictions for new instances.

\n", "

Estimation of house prices, product recommendations, street sign detection, credit default prediction and fraud detection:\n", "All these examples have in common that they can be solved by machine learning.\n", "The tasks are different, but the approach is the same:
\n", "Step 1: Data collection.\n", "The more, the better.\n", "The data must contain the outcome you want to predict and additional information from which to make the prediction.\n", "For a street sign detector (“Is there a street sign in the image?”), you would collect street images and label whether a street sign is visible or not.\n", "For a credit default predictor, you need past data on actual loans, information on whether the customers were in default with their loans, and data that will help you make predictions, such as income, past credit defaults, and so on.\n", "For an automatic house value estimator program, you could collect data from past house sales and information about the real estate such as size, location, and so on.
\n", "Step 2: Enter this information into a machine learning algorithm that generates a sign detector model, a credit rating model or a house value estimator.
\n", "Step 3: Use model with new data.\n", "Integrate the model into a product or process, such as a self-driving car, a credit application process or a real estate marketplace website.

\n", "

Machines surpass humans in many tasks, such as playing chess (or more recently Go) or predicting the weather.\n", "Even if the machine is as good as a human or a bit worse at a task, there remain great advantages in terms of speed, reproducibility and scaling.\n", "A once implemented machine learning model can complete a task much faster than humans, reliably delivers consistent results and can be copied infinitely.\n", "Replicating a machine learning model on another machine is fast and cheap.\n", "The training of a human for a task can take decades (especially when they are young) and is very costly.\n", "A major disadvantage of using machine learning is that insights about the data and the task the machine solves is hidden in increasingly complex models.\n", "You need millions of numbers to describe a deep neural network, and there is no way to understand the model in its entirety.\n", "Other models, such as the random forest, consist of hundreds of decision trees that “vote” for predictions.\n", "To understand how the decision was made, you would have to look into the votes and structures of each of the hundreds of trees.\n", "That just does not work no matter how clever you are or how good your working memory is.\n", "The best performing models are often blends of several models (also called ensembles) that cannot be interpreted, even if each single model could be interpreted.\n", "If you focus only on performance, you will automatically get more and more opaque models.\n", "\n", "The winning models on machine learning competitions are often ensembles of models or very complex models such as boosted trees or deep neural networks.

\n", "\n", "
\n", "
\n", "
\n", "
\n", "
\n", "\n", "\n", "
\n", "
\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "soup = BeautifulSoup(r.text)\n", "\n", "soup" ] }, { "cell_type": "markdown", "metadata": { "id": "-kBPILKtfdmq" }, "source": [ "That seems more or less the same but now a little easier to read. What has BeautifulSoup done? Now, we can use the `.find` method to pull out individual page elements, by [tag, class, or id](https://en.wikipedia.org/wiki/HTML#Elements). Here we will just grab the first paragraph element:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "executionInfo": { "elapsed": 272, "status": "ok", "timestamp": 1687981821253, "user": { "displayName": "Myles Harrison", "userId": "02225822412878142066" }, "user_tz": 240 }, "id": "kCWyxYwXfdmr", "outputId": "102d5b84-e944-4de0-eb93-8384cb505e8a" }, "outputs": [ { "data": { "text/plain": [ "

Machine learning is a set of methods that computers use to make and improve predictions or behaviors based on data.

" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "soup.find('p')" ] }, { "cell_type": "markdown", "metadata": { "id": "JSXst7ecfdmr" }, "source": [ "Furthermore, we can use `find_all` to return all elements of a given type in the page as an array and iterate over it, and pull out only the text of each paragraph using the `.text` attribute. Let's look at the first 3 paragraphs:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "executionInfo": { "elapsed": 33, "status": "ok", "timestamp": 1687981821254, "user": { "displayName": "Myles Harrison", "userId": "02225822412878142066" }, "user_tz": 240 }, "id": "wt647SWLfdms", "outputId": "a820e392-80e1-4179-f5a4-ad89e7854539" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Machine learning is a set of methods that computers use to make and improve predictions or behaviors based on data.\n", "For example, to predict the value of a house, the computer would learn patterns from past house sales.\n", "The book focuses on supervised machine learning, which covers all prediction problems where we have a dataset for which we already know the outcome of interest (e.g. past house prices) and want to learn to predict the outcome for new data.\n", "Excluded from supervised learning are for example clustering tasks (= unsupervised learning) where we do not have a specific outcome of interest, but want to find clusters of data points.\n", "Also excluded are things like reinforcement learning, where an agent learns to optimize a certain reward by acting in an environment (e.g. a computer playing Tetris).\n", "The goal of supervised learning is to learn a predictive model that maps features of the data (e.g. house size, location, floor type, …) to an output (e.g. house price).\n", "If the output is categorical, the task is called classification, and if it is numerical, it is called regression.\n", "The machine learning algorithm learns a model by estimating parameters (like weights) or learning structures (like trees).\n", "The algorithm is guided by a score or loss function that is minimized.\n", "In the house value example, the machine minimizes the difference between the estimated house price and the predicted price.\n", "A fully trained machine learning model can then be used to make predictions for new instances.\n" ] } ], "source": [ "# Check first 3 elements\n", "for elem in soup.find_all(\"p\")[0:2]:\n", " print(elem.text)" ] }, { "cell_type": "markdown", "metadata": { "id": "A0dXcuNRfdms" }, "source": [ "Great, now let's join it all together, and replace the newline characters with spaces, to create one giant string of text:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "executionInfo": { "elapsed": 29, "status": "ok", "timestamp": 1687981821254, "user": { "displayName": "Myles Harrison", "userId": "02225822412878142066" }, "user_tz": 240 }, "id": "8rlbYAkvfdmt", "outputId": "d5c50c6d-354b-4846-e2f9-9a86cec0db74", "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Machine learning is a set of methods that computers use to make and improve predictions or behaviors based on data. For example, to predict the value of a house, the computer would learn patterns from past house sales. The book focuses on supervised machine learning, which covers all prediction problems where we have a dataset for which we already know the outcome of interest (e.g. past house prices) and want to learn to predict the outcome for new data. Excluded from supervised learning are for example clustering tasks (= unsupervised learning) where we do not have a specific outcome of interest, but want to find clusters of data points. Also excluded are things like reinforcement learning, where an agent learns to optimize a certain reward by acting in an environment (e.g. a computer playing Tetris). The goal of supervised learning is to learn a predictive model that maps features of the data (e.g. house size, location, floor type, …) to an output (e.g. house price). If the output is categorical, the task is called classification, and if it is numerical, it is called regression. The machine learning algorithm learns a model by estimating parameters (like weights) or learning structures (like trees). The algorithm is guided by a score or loss function that is minimized. In the house value example, the machine minimizes the difference between the estimated house price and the predicted price. A fully trained machine learning model can then be used to make predictions for new instances. Estimation of house prices, product recommendations, street sign detection, credit default prediction and fraud detection: All these examples have in common that they can be solved by machine learning. The tasks are different, but the approach is the same: Step 1: Data collection. The more, the better. The data must contain the outcome you want to predict and additional information from which to make the prediction. For a street sign detector (“Is there a street sign in the image?”), you would collect street images and label whether a street sign is visible or not. For a credit default predictor, you need past data on actual loans, information on whether the customers were in default with their loans, and data that will help you make predictions, such as income, past credit defaults, and so on. For an automatic house value estimator program, you could collect data from past house sales and information about the real estate such as size, location, and so on. Step 2: Enter this information into a machine learning algorithm that generates a sign detector model, a credit rating model or a house value estimator. Step 3: Use model with new data. Integrate the model into a product or process, such as a self-driving car, a credit application process or a real estate marketplace website. Machines surpass humans in many tasks, such as playing chess (or more recently Go) or predicting the weather. Even if the machine is as good as a human or a bit worse at a task, there remain great advantages in terms of speed, reproducibility and scaling. A once implemented machine learning model can complete a task much faster than humans, reliably delivers consistent results and can be copied infinitely. Replicating a machine learning model on another machine is fast and cheap. The training of a human for a task can take decades (especially when they are young) and is very costly. A major disadvantage of using machine learning is that insights about the data and the task the machine solves is hidden in increasingly complex models. You need millions of numbers to describe a deep neural network, and there is no way to understand the model in its entirety. Other models, such as the random forest, consist of hundreds of decision trees that “vote” for predictions. To understand how the decision was made, you would have to look into the votes and structures of each of the hundreds of trees. That just does not work no matter how clever you are or how good your working memory is. The best performing models are often blends of several models (also called ensembles) that cannot be interpreted, even if each single model could be interpreted. If you focus only on performance, you will automatically get more and more opaque models. The winning models on machine learning competitions are often ensembles of models or very complex models such as boosted trees or deep neural networks. \n" ] } ], "source": [ "text = \"\"\n", "\n", "for paragraph in soup.find_all(\"p\"):\n", " text += paragraph.text.replace('\\n', ' ') + ' '\n", "\n", "print(text)" ] }, { "cell_type": "markdown", "metadata": { "id": "SndgFOaufdmt" }, "source": [ "We now have a scraped text data from a website! Doing so for more complicated pages or programatically over many pages can be accomplished with more code and inspecting the different pages' structure. The difficulty or ease of doing so will depend upon how the page is site is structured and page code." ] }, { "cell_type": "markdown", "metadata": { "id": "Gz9_m9W5fdmt" }, "source": [ "## Data Preprocessing\n", "\n", "We have now acquired some text. Before using this text in an ML application of NLP, we first need to preprocess the data.\n", "\n", "As outlined in the slides, major steps in preprocessing text are:\n", "- Normalization (addressing case, removing punctuation and stop words, stemming or lemmatization)\n", "- Tokenization (breaking up into individual units of language, usually words)\n", "- Vectorization (converting tokens to structured numeric data)" ] }, { "cell_type": "markdown", "metadata": { "id": "UM5xq1twfdmu" }, "source": [ "### Normalization\n", "\n", "There are a few things we need to do here: *addressing case, removing punctuation, and stemming or lemmatization*. For simplicity's sake, we will not expand contractions (don't, won't, can't, etc.) though this would be another normalization step. We will also only try the simpler technique of stemming, though there are lemmatizers built in to packages such as [nltk](https://www.nltk.org/_modules/nltk/stem/wordnet.html) and [TextBlob](https://textblob.readthedocs.io/en/dev/quickstart.html#words-inflection-and-lemmatization).\n", "\n", "To standardize the case, we simply convert everything to lowercase:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 70 }, "executionInfo": { "elapsed": 26, "status": "ok", "timestamp": 1687981821255, "user": { "displayName": "Myles Harrison", "userId": "02225822412878142066" }, "user_tz": 240 }, "id": "OQpIAHZifdmu", "outputId": "b8dbccea-a218-4f13-c470-5aafd9bd2e2b" }, "outputs": [ { "data": { "application/vnd.google.colaboratory.intrinsic+json": { "type": "string" }, "text/plain": [ "'machine learning is a set of methods that computers use to make and improve predictions or behaviors based on data. for example, to predict the value of a house, the computer would learn patterns from past house sales. the book focuses on supervised machine learning, which covers all prediction problems where we have a dataset for which we already know the outcome of interest (e.g.\\xa0past house prices) and want to learn to predict the outcome for new data. excluded from supervised learning are for'" ] }, "execution_count": 16, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Convert to lower case\n", "text = text.lower()\n", "text[0:500]" ] }, { "cell_type": "markdown", "metadata": { "id": "i-5ifOJkfdmv" }, "source": [ "It appears there are also some unicode characters mixed in there, which is never good. Dealing with special characters and different text encodings can be one of the challenge parts of doing NLP. We will change the encoding to ASCII to address these:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 70 }, "executionInfo": { "elapsed": 25, "status": "ok", "timestamp": 1687981821256, "user": { "displayName": "Myles Harrison", "userId": "02225822412878142066" }, "user_tz": 240 }, "id": "lpDjQLslfdmv", "outputId": "3fc86e99-db04-4b26-ea58-8ef5949912bb" }, "outputs": [ { "data": { "application/vnd.google.colaboratory.intrinsic+json": { "type": "string" }, "text/plain": [ "'machine learning is a set of methods that computers use to make and improve predictions or behaviors based on data. for example, to predict the value of a house, the computer would learn patterns from past house sales. the book focuses on supervised machine learning, which covers all prediction problems where we have a dataset for which we already know the outcome of interest (e.g.past house prices) and want to learn to predict the outcome for new data. excluded from supervised learning are for '" ] }, "execution_count": 17, "metadata": {}, "output_type": "execute_result" } ], "source": [ "text = text.encode('ASCII', errors='ignore').decode()\n", "text[0:500]" ] }, { "cell_type": "markdown", "metadata": { "id": "Vu-77Ypsfdmw" }, "source": [ "That's better, we can see the special characters like `\\xa0` that were present before are now gone. Next we will remove all punctuation. Fortunately, all the punctuation characters are contained in a string stored in the python `string` base module:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "executionInfo": { "elapsed": 23, "status": "ok", "timestamp": 1687981821257, "user": { "displayName": "Myles Harrison", "userId": "02225822412878142066" }, "user_tz": 240 }, "id": "VYJRB8qmfdm3", "outputId": "393bd80c-db69-40b3-d758-9984125e24fc" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "!\"#$%&'()*+,-./:;<=>?@[\\]^_`{|}~\n" ] } ], "source": [ "from string import punctuation\n", "\n", "print(punctuation)" ] }, { "cell_type": "markdown", "metadata": { "id": "VaTLP6n1fdm3" }, "source": [ "We can now iterate over each punctuation character, and update the text, replacing it with the empty string `''`:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "ppvgUAKMfdm4" }, "outputs": [], "source": [ "for punctuation_mark in punctuation:\n", " text = text.replace(punctuation_mark, '')" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 122 }, "executionInfo": { "elapsed": 18, "status": "ok", "timestamp": 1687981821258, "user": { "displayName": "Myles Harrison", "userId": "02225822412878142066" }, "user_tz": 240 }, "id": "xRzf8fczfdm4", "outputId": "7d18514e-75bd-4923-faf7-ff229a544e55" }, "outputs": [ { "data": { "application/vnd.google.colaboratory.intrinsic+json": { "type": "string" }, "text/plain": [ "'machine learning is a set of methods that computers use to make and improve predictions or behaviors based on data for example to predict the value of a house the computer would learn patterns from past house sales the book focuses on supervised machine learning which covers all prediction problems where we have a dataset for which we already know the outcome of interest egpast house prices and want to learn to predict the outcome for new data excluded from supervised learning are for example clustering tasks unsupervised learning where we do not have a specific outcome of interest but want to find clusters of data points also excluded are things like reinforcement learning where an agent learns to optimize a certain reward by acting in an environment ega computer playing tetris the goal of supervised learning is to learn a predictive model that maps features of the data eghouse size location floor type to an output eghouse price if the output is categorical the task is called classi'" ] }, "execution_count": 20, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Check the result\n", "text[0:1000]" ] }, { "cell_type": "markdown", "metadata": { "id": "-f_rCGu2fdm5" }, "source": [ "We can already see some strange things are happening, such as the *e.g.* getting folded into the words \"past\" and \"house\" to create the tokens \"egpast\" and \"eghouse\". Preprocessing text is not an exact science... we will proceed as is for now, though there perhaps could have been better ways to tokenize or deal with problematic portions of this text such as abbreviations." ] }, { "cell_type": "markdown", "metadata": { "id": "CcBunenJfdm5" }, "source": [ "### Stemming\n", "Stemming and lemmatization are built into the very powerful [nltk toolkit](https://www.nltk.org/). Here we choose to do simple stemming using the `SnowBallStemmer` (a particular type of stemming algorithm):" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 36 }, "executionInfo": { "elapsed": 1878, "status": "ok", "timestamp": 1687981823119, "user": { "displayName": "Myles Harrison", "userId": "02225822412878142066" }, "user_tz": 240 }, "id": "Jco34Xp-fdm5", "outputId": "4ef2482d-d309-499e-e025-ea43cc0bd6bf" }, "outputs": [ { "data": { "application/vnd.google.colaboratory.intrinsic+json": { "type": "string" }, "text/plain": [ "'run'" ] }, "execution_count": 21, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from nltk.stem import SnowballStemmer\n", "\n", "# Instantiate stemmer for english\n", "sbstem = SnowballStemmer('english')\n", "\n", "# Check\n", "sbstem.stem('running')" ] }, { "cell_type": "markdown", "metadata": { "id": "lC5socchfdm6" }, "source": [ "Stemmers in `nltk` operation on individual tokens, so we must iterate over the freeform text, then join everything back together again:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "bdgtcAYLfdm6" }, "outputs": [], "source": [ "# Create an empty list for the stemmed words\n", "stemmed_words = list()\n", "\n", "# Iterate over each word and stem and add to new string\n", "for word in text.split(' '):\n", " stemmed_words.append(sbstem.stem(word))\n", "\n", "# Join it all back together and remove any repeated or extraneous spaces\n", "text = ' '.join(stemmed_words).strip()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 70 }, "executionInfo": { "elapsed": 255, "status": "ok", "timestamp": 1687981823368, "user": { "displayName": "Myles Harrison", "userId": "02225822412878142066" }, "user_tz": 240 }, "id": "XspuoPN1fdm7", "outputId": "e8f59f05-cbea-40f1-ef11-0f9bd23ab11e" }, "outputs": [ { "data": { "application/vnd.google.colaboratory.intrinsic+json": { "type": "string" }, "text/plain": [ "'machin learn is a set of method that comput use to make and improv predict or behavior base on data for exampl to predict the valu of a hous the comput would learn pattern from past hous sale the book focus on supervis machin learn which cover all predict problem where we have a dataset for which we alreadi know the outcom of interest egpast hous price and want to learn to predict the outcom for new data exclud from supervis learn are for exampl cluster task unsupervis learn where we do not hav'" ] }, "execution_count": 23, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Check\n", "text[0:500]" ] }, { "cell_type": "markdown", "metadata": { "id": "N2RUv2R1fdm7" }, "source": [ "We can see that words like *machine* have been stemmed to *machin*, and *improve* to *improv*, and so on, so we appear to have applied stemming correctly. This is enough normalization for now, and we can move on to tokenizing and vectorizing our text." ] }, { "cell_type": "markdown", "metadata": { "id": "PUmbRzM8fdm7" }, "source": [ "### Tokenization" ] }, { "cell_type": "markdown", "metadata": { "id": "gYEpKgSFfdm8" }, "source": [ "Our approach for tokenization could be as simple as splitting on whitespace. As we saw above, we actually did this step and then undid it, as the `nltk` stemmer works on individual words. Alternatively, we could have applied stemming *after* or as part of tokenization, as the different steps in text preprocessing are not necessarily always in a particular order depending upon implementation).\n", "\n", "Splitting on whitespace is as simple as using the `.split()` method in base python built in to any string variable:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "executionInfo": { "elapsed": 21, "status": "ok", "timestamp": 1687981823370, "user": { "displayName": "Myles Harrison", "userId": "02225822412878142066" }, "user_tz": 240 }, "id": "SxB3oiIzfdm8", "outputId": "744f5407-f1b3-444e-f9ef-2557d4cc474d" }, "outputs": [ { "data": { "text/plain": [ "['machin', 'learn', 'is', 'a', 'set', 'of', 'method', 'that', 'comput', 'use']" ] }, "execution_count": 24, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Tokenize and show first 10 tokens\n", "text.split(' ')[0:10]" ] }, { "cell_type": "markdown", "metadata": { "id": "j-9ONzIzfdm8" }, "source": [ "More sophisticated approaches for tokenization exist. We actually do not need do this step manually, as it is included in the vectorization step in code as part of `scikit-learn` as we will see below." ] }, { "cell_type": "markdown", "metadata": { "id": "NXJ6Hy-gfdm9" }, "source": [ "### Vectorization\n", "\n", "There are two standard types of vectorization used in traditional NLP: *count vectorization* and *term frequency - inverse document frequency (tf-idf)* vectorization. Binary (\"One-hot\") encoding with a boolean (0/1) flag for word occurrence in each document can also be done, though this is less common.\n", "\n", "The two former vectorization methods are implemented in `scikit-learn` in the `feature_extraction.text` submodule and we can apply as below:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "RYndP7OUfdm9" }, "outputs": [], "source": [ "from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer\n", "\n", "# Instatiate, fit and transform - Count vectorization\n", "cv = CountVectorizer()\n", "count_vectorized = cv.fit_transform([text])\n", "\n", "# Instatiate, fit and transform - TF-IDF vectorization\n", "tfidf = TfidfVectorizer()\n", "tfidf_vectorized = tfidf.fit_transform([text])" ] }, { "cell_type": "markdown", "metadata": { "id": "31Gqguvzfdm-" }, "source": [ "Let's take a look at the outputs:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "executionInfo": { "elapsed": 18, "status": "ok", "timestamp": 1687981823372, "user": { "displayName": "Myles Harrison", "userId": "02225822412878142066" }, "user_tz": 240 }, "id": "YZ7CZEbHfdm_", "outputId": "7a788dff-3eb2-462c-db07-b0044aed8110" }, "outputs": [ { "data": { "text/plain": [ "<1x263 sparse matrix of type ''\n", "\twith 263 stored elements in Compressed Sparse Row format>" ] }, "execution_count": 26, "metadata": {}, "output_type": "execute_result" } ], "source": [ "count_vectorized" ] }, { "cell_type": "markdown", "metadata": { "id": "MrvFeFjTfdm_" }, "source": [ "This is a sparse matrix - the *document-term matrix* - with each feature (column) being a token that appeared in the documents (rows), so there are 263 unique tokens in our single document of text. This can be cast into a dense array and we can pass the tokens as the column names using pandas:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 110 }, "executionInfo": { "elapsed": 260, "status": "ok", "timestamp": 1687981823617, "user": { "displayName": "Myles Harrison", "userId": "02225822412878142066" }, "user_tz": 240 }, "id": "smi41HIHfdnA", "outputId": "bb5805f6-48fd-44d6-c26a-69b350b50b3e" }, "outputs": [ { "data": { "text/html": [ "\n", "
\n", "
\n", "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
aboutactactualadditadvantagagentalgorithmallalreadialso...whichwillwinwithworkworswouldyouyoungyour
02111113212...32122131011
\n", "

1 rows × 263 columns

\n", "
\n", " \n", " \n", " \n", "\n", " \n", "
\n", "
\n", " " ], "text/plain": [ " about act actual addit advantag agent algorithm all alreadi also \\\n", "0 2 1 1 1 1 1 3 2 1 2 \n", "\n", " ... which will win with work wors would you young your \n", "0 ... 3 2 1 2 2 1 3 10 1 1 \n", "\n", "[1 rows x 263 columns]" ] }, "execution_count": 27, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import pandas as pd\n", "count_df = pd.DataFrame(count_vectorized.todense(), columns=cv.get_feature_names_out())\n", "\n", "count_df.head()" ] }, { "cell_type": "markdown", "metadata": { "id": "EXWOdb3WfdnB" }, "source": [ "Great, we now have a count of the number of occurrences of each token in our text!\n", "\n", "Contrast this with the floating point numbers from tf-idf vectorization:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 174 }, "executionInfo": { "elapsed": 11, "status": "ok", "timestamp": 1687981823618, "user": { "displayName": "Myles Harrison", "userId": "02225822412878142066" }, "user_tz": 240 }, "id": "0t5xrZrUfdnB", "outputId": "b9a6d8c9-3efd-4c8e-f9b3-05b86efb0269" }, "outputs": [ { "data": { "text/html": [ "\n", "
\n", "
\n", "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
aboutactactualadditadvantagagentalgorithmallalreadialso...whichwillwinwithworkworswouldyouyoungyour
00.025570.0127850.0127850.0127850.0127850.0127850.0383550.025570.0127850.02557...0.0383550.025570.0127850.025570.025570.0127850.0383550.1278480.0127850.012785
\n", "

1 rows × 263 columns

\n", "
\n", " \n", " \n", " \n", "\n", " \n", "
\n", "
\n", " " ], "text/plain": [ " about act actual addit advantag agent algorithm \\\n", "0 0.02557 0.012785 0.012785 0.012785 0.012785 0.012785 0.038355 \n", "\n", " all alreadi also ... which will win with \\\n", "0 0.02557 0.012785 0.02557 ... 0.038355 0.02557 0.012785 0.02557 \n", "\n", " work wors would you young your \n", "0 0.02557 0.012785 0.038355 0.127848 0.012785 0.012785 \n", "\n", "[1 rows x 263 columns]" ] }, "execution_count": 28, "metadata": {}, "output_type": "execute_result" } ], "source": [ "tfidf_df = pd.DataFrame(tfidf_vectorized.todense(), columns=tfidf.get_feature_names_out())\n", "\n", "tfidf_df.head()" ] }, { "cell_type": "markdown", "metadata": { "id": "qoTH7PHYfdnC" }, "source": [ "From our count vectorization, we now have a count of each token. Normally our document-term matrix would have many rows, one for each document in our corpus. Nonetheless, we can find the most frequently occurring tokens in the text we scraped by doing a simple sort and making a bar chart:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 472 }, "executionInfo": { "elapsed": 538, "status": "ok", "timestamp": 1687981824147, "user": { "displayName": "Myles Harrison", "userId": "02225822412878142066" }, "user_tz": 240 }, "id": "lF_Wx2UifdnC", "outputId": "81999c62-a024-4aae-a2c5-b13fcb42467e" }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlQAAAHHCAYAAAB5gsZZAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABMa0lEQVR4nO3deVhV5f7//9cGYcuMKAkozvOAmlNqHnEoHHMoKzM1NS1zwqm0QcUsPVqfzE5ZWkftnNJTaZmZOSVW5GxOSWYm4YCRmiCSqHD//ujH/roFFVzqBnw+rmtfF3ute6/1vtcCeXmvey1sxhgjAAAAXDc3VxcAAABQ2BGoAAAALCJQAQAAWESgAgAAsIhABQAAYBGBCgAAwCICFQAAgEUEKgAAAIsIVAAAABYRqADc1hISEmSz2bRgwQJXl1JgREZGKjIy0tVlXLfJkyfLZrPpxIkTri4FtxECFZBHCxYskM1mk81m03fffZdjvTFG4eHhstls6ty5802p4dixY5o8ebJ27tyZp/aX1nz5a/z48TelxoLqww8/1KxZs27Z/n788Uc9+uijKlOmjOx2u8LCwtS7d2/9+OOPt6yGwiQ7BF3rVZiDHoq2Yq4uAChsihcvrg8//FB333230/INGzboyJEjstvtN23fx44dU0xMjCpUqKD69evn+XNTpkxRxYoVnZbVqVPnBldXsH344Yfau3evoqOjb/q+li5dql69eikoKEgDBw5UxYoVlZCQoPfee0+ffPKJFi9erO7du9/0Oq7X6tWrb/k+e/TooSpVqjjep6WlaciQIerevbt69OjhWF66dOlbXhuQFwQqIJ86duyojz/+WLNnz1axYv/vR+jDDz9Uw4YNC+Rlhg4dOqhRo0Z5anvu3Dl5enrKzY0B7Otx8OBB9enTR5UqVdI333yj4OBgx7qRI0eqZcuW6tOnj3bv3q1KlSq5sNKc0tPT5e3tLU9Pz1u+74iICEVERDjenzhxQkOGDFFERIQeffTRW14PkF/8iwnkU69evXTy5EmtWbPGsez8+fP65JNP9Mgjj+T6mbNnz2rMmDEKDw+X3W5X9erV9corr8gY49RuzZo1uvvuuxUYGChfX19Vr15dzz77rCQpNjZWjRs3liT179/fcQnEytyf2NhY2Ww2LV68WM8//7zKlCkjb29vpaamSpI2b96s9u3bKyAgQN7e3mrVqpXi4uJybOe7775T48aNVbx4cVWuXFnvvPOO4xJOtqvNVbLZbJo8ebLTsqNHj2rAgAEqXbq07Ha7ateurX//+9+51v/RRx/ppZdeUtmyZVW8eHG1bdtWv/zyi6NdZGSkVqxYod9++81x3CpUqJDrMZk/f75sNpt++OGHHOtefvllubu76+jRo1c6pJo5c6bS09M1d+5cpzAlSaVKldI777yjs2fPasaMGTn6O3DgQIWFhclut6tixYoaMmSIzp8/72hz+vRpjRo1ShUqVJDdblfZsmXVt29fR4jPvsSbkJCQ63GKjY11OiZ16tTR9u3b9Y9//EPe3t6O77XL51Dl9Thne/PNN1WpUiV5eXmpSZMm+vbbb2/YvKyvv/5aLVu2lI+PjwIDA9W1a1fFx8df83O//fabqlSpojp16uj333+X9PfxjI6OdvxcVqlSRf/85z+VlZXl+Fz29+0rr7yiuXPnqnLlyrLb7WrcuLG2bt3qtI/jx4+rf//+Klu2rOx2u0JDQ9W1a9cc5wNFEyNUQD5VqFBBzZo106JFi9ShQwdJ0sqVK5WSkqKHH35Ys2fPdmpvjNF9992n9evXa+DAgapfv75WrVqlcePG6ejRo3rttdck/T3npnPnzoqIiNCUKVNkt9v1yy+/OAJMzZo1NWXKFE2cOFGDBw9Wy5YtJUnNmze/Zs0pKSk5Rs5KlSrl+PrFF1+Up6enxo4dq4yMDHl6eurrr79Whw4d1LBhQ02aNElubm6aP3++2rRpo2+//VZNmjSRJO3Zs0f33nuvgoODNXnyZF28eFGTJk2ydGnm999/11133SWbzaZhw4YpODhYK1eu1MCBA5Wamprjst306dPl5uamsWPHKiUlRTNmzFDv3r21efNmSdJzzz2nlJQUHTlyxHG8fX19c933Aw88oKFDh+qDDz5QgwYNnNZ98MEHioyMVJkyZa5Y+/Lly1WhQgXH+bncP/7xD1WoUEErVqxwLDt27JiaNGmi06dPa/DgwapRo4aOHj2qTz75ROnp6fL09FRaWppatmyp+Ph4DRgwQHfeeadOnDihzz//XEeOHHE6n3l18uRJdejQQQ8//LAeffTRa56zax1nSZozZ46GDRumli1batSoUUpISFC3bt1UokQJlS1bNt81Xmrt2rXq0KGDKlWqpMmTJ+uvv/7SG2+8oRYtWmjHjh1XDMkHDx5UmzZtFBQUpDVr1qhUqVJKT09Xq1atdPToUT3xxBMqV66cvv/+e02YMEFJSUk55tt9+OGHOnPmjJ544gnZbDbNmDFDPXr00K+//ioPDw9J0v33368ff/xRw4cPV4UKFZScnKw1a9YoMTHxirWhCDEA8mT+/PlGktm6dav517/+Zfz8/Ex6eroxxpiePXua1q1bG2OMKV++vOnUqZPjc5999pmRZKZOneq0vQceeMDYbDbzyy+/GGOMee2114wk88cff1yxhq1btxpJZv78+fmqObeXMcasX7/eSDKVKlVy9MUYY7KyskzVqlVNVFSUycrKcixPT083FStWNPfcc49jWbdu3Uzx4sXNb7/95li2b98+4+7ubi79J+bQoUNXrF2SmTRpkuP9wIEDTWhoqDlx4oRTu4cfftgEBAQ4as2uv2bNmiYjI8PR7vXXXzeSzJ49exzLOnXqZMqXL59j37nV1atXLxMWFmYyMzMdy3bs2HHNY3/69GkjyXTt2vWKbYwx5r777jOSTGpqqjHGmL59+xo3NzezdevWHG2zj//EiRONJLN06dIrtsk+34cOHXJan32c1q9f71jWqlUrI8m8/fbbObbXqlUr06pVqxyfv9ZxzsjIMCVLljSNGzc2Fy5ccLRbsGCBkeS0zWv5448/cnxf1K9f39xxxx3m5MmTjmW7du0ybm5upm/fvo5lkyZNcvwsxcfHm7CwMNO4cWNz6tQpR5sXX3zR+Pj4mJ9//tlpv+PHjzfu7u4mMTHRGPP/vj9Klizp9Plly5YZSWb58uXGGGP+/PNPI8nMnDkzz31E0cIlP+A6PPjgg/rrr7/0xRdf6MyZM/riiy+ueLnvyy+/lLu7u0aMGOG0fMyYMTLGaOXKlZKkwMBASdKyZcucLjncCG+++abWrFnj9LpUv3795OXl5Xi/c+dOHThwQI888ohOnjypEydO6MSJEzp79qzatm2rb775RllZWcrMzNSqVavUrVs3lStXzvH5mjVrKioq6rpqNcZoyZIl6tKli4wxjn2fOHFCUVFRSklJ0Y4dO5w+079/f6d5P9mjQ7/++ut11dC3b18dO3ZM69evdyz74IMP5OXlpfvvv/+Knztz5owkyc/P76rbz16fmpqqrKwsffbZZ+rSpUuu89yyL5suWbJE9erVy3Uy+6WXVvPDbrerf//+eW5/reO8bds2nTx5UoMGDXKaX9i7d2+VKFHiumrMlpSUpJ07d+qxxx5TUFCQY3lERITuueceffnllzk+s3fvXrVq1UoVKlTQ2rVrnWr4+OOP1bJlS5UoUcLpe6xdu3bKzMzUN99847Sthx56yOnzl/fdy8tLnp6eio2N1Z9//mmpryicuOQHXIfg4GC1a9dOH374odLT05WZmakHHngg17a//fabwsLCcvySrVmzpmO99Pc/2O+++64ef/xxjR8/Xm3btlWPHj30wAMPWJ4g3qRJk6tOSr/8DsADBw5I+jtoXUlKSooyMjL0119/qWrVqjnWV69ePddfctfyxx9/6PTp05o7d67mzp2ba5vk5GSn95eGOUmOX3zX+4vtnnvuUWhoqD744AO1bdtWWVlZWrRokbp27XrVsJS9LjtYXcmlweuPP/5QamrqNe+6PHjw4FXD3PUoU6ZMviagX+s4Z38vX3q3niQVK1bM8iWv7G1Xr149x7qaNWtq1apVOnv2rHx8fBzLu3TpotKlS2vVqlU5LvEeOHBAu3fvzjHPLVt+v8fsdrv++c9/asyYMSpdurTuuusude7cWX379lVISEg+e4vCiEAFXKdHHnlEgwYN0vHjx9WhQwfHCNP18vLy0jfffKP169drxYoV+uqrr/S///1Pbdq00erVq+Xu7n5jCr/Cvi+VPUI2c+bMKz6ewdfXVxkZGXnex5VGUTIzM3Pd96OPPnrFQHfp3WCSrnhszGWT/vPK3d1djzzyiObNm6e33npLcXFxOnbs2DXvNgsICFBoaKh279591Xa7d+9WmTJl5O/vr7/++uu6asxNXo9xtsvP+7Xc6ON8s91///1auHChPvjgAz3xxBNO67KysnTPPffo6aefzvWz1apVc3qfl75HR0erS5cu+uyzz7Rq1Sq98MILmjZtmr7++usc8/FQ9BCogOvUvXt3PfHEE9q0aZP+97//XbFd+fLltXbtWp05c8ZpdOOnn35yrM/m5uamtm3bqm3btvq///s/vfzyy3ruuee0fv16tWvX7rov7eRX5cqVJUn+/v5q167dFdsFBwfLy8vLMaJ1qf379zu9z/4f/enTp52WZ488XLpNPz8/ZWZmXnXf+ZXfY9e3b1+9+uqrWr58uVauXKng4OA8Xcbs3Lmz5s2bp++++y7Hs8ok6dtvv1VCQoLjF3xwcLD8/f21d+/eq263cuXK12yT12N8s2R/L//yyy9q3bq1Y/nFixeVkJCQIwhfz7Yv/76S/v5ZKlWqlNPolPT3fwiKFSump556Sn5+fk6X5StXrqy0tLQb+j2Wvd0xY8ZozJgxOnDggOrXr69XX31V//3vf2/oflDwMIcKuE6+vr6aM2eOJk+erC5dulyxXceOHZWZmal//etfTstfe+012Ww2x52Cp06dyvHZ7NGh7JGg7F8Yl//CvNEaNmyoypUr65VXXlFaWlqO9X/88Yekv//XHhUVpc8++0yJiYmO9fHx8Vq1apXTZ/z9/VWqVKkcc1Peeustp/fu7u66//77tWTJklwDRPa+88vHx0cpKSl5bp/9XKR3331XS5Ys0cMPP+w0L+hKxo0bJy8vLz3xxBM6efKk07pTp07pySeflLe3t8aNGyfp7xDdrVs3LV++XNu2bcuxvewRkPvvv1+7du3Sp59+esU22UH40mOcmZl5xUunN1qjRo1UsmRJzZs3TxcvXnQs/+CDDyzPKwoNDVX9+vW1cOFCp+//vXv3avXq1erYsWOOz9hsNs2dO1cPPPCA+vXrp88//9yx7sEHH9TGjRtzfJ9Kf/98XVp/XqSnp+vcuXNOyypXriw/P798jeSi8GKECrDganOMsnXp0kWtW7fWc889p4SEBNWrV0+rV6/WsmXLFB0d7fglOGXKFH3zzTfq1KmTypcvr+TkZL311lsqW7asY6SjcuXKCgwM1Ntvvy0/Pz/5+PioadOmOeZAWeXm5qZ3331XHTp0UO3atdW/f3+VKVNGR48e1fr16+Xv76/ly5dLkmJiYvTVV1+pZcuWeuqpp3Tx4kW98cYbql27do5LX48//rimT5+uxx9/XI0aNdI333yjn3/+Ocf+p0+frvXr16tp06YaNGiQatWqpVOnTmnHjh1au3ZtruHzWho2bKj//e9/Gj16tBo3bixfX9+rBmHp71GqsWPHSlKeHy5ZtWpVLVy4UL1791bdunVzPCn9xIkTWrRokeO8S38/32r16tVq1aqVBg8erJo1ayopKUkff/yxvvvuOwUGBmrcuHH65JNP1LNnTw0YMEANGzbUqVOn9Pnnn+vtt99WvXr1VLt2bd11112aMGGCTp06paCgIC1evDjf4eB6eXp6avLkyRo+fLjatGmjBx98UAkJCVqwYIEqV65seYR15syZ6tChg5o1a6aBAwc6HpsQEBCQ4zlm2dzc3PTf//5X3bp104MPPqgvv/xSbdq00bhx4/T555+rc+fOeuyxx9SwYUOdPXtWe/bs0SeffKKEhIR8PYri559/Vtu2bfXggw+qVq1aKlasmD799FP9/vvvevjhhy31G4WE624wBAqXSx+bcDWXPzbBGGPOnDljRo0aZcLCwoyHh4epWrWqmTlzptMjCdatW2e6du1qwsLCjKenpwkLCzO9evXKcVv3smXLTK1atUyxYsWueRv/tWrOvh3+448/znX9Dz/8YHr06GFKlixp7Ha7KV++vHnwwQfNunXrnNpt2LDBNGzY0Hh6eppKlSqZt99+23Hr+qXS09PNwIEDTUBAgPHz8zMPPvigSU5OznF7vDHG/P7772bo0KEmPDzceHh4mJCQENO2bVszd+7ca9af26MQ0tLSzCOPPGICAwONJMcjFK72OIekpCTj7u5uqlWrluvxuZrdu3ebXr16mdDQUEf9vXr1cnqUw6V+++0307dvXxMcHGzsdrupVKmSGTp0qNNjCk6ePGmGDRtmypQpYzw9PU3ZsmVNv379nB4vcfDgQdOuXTtjt9tN6dKlzbPPPmvWrFmT62MTateunWstV3psQl6OszHGzJ4925QvX97Y7XbTpEkTExcXZxo2bGjat2+fx6OX+2MTjDFm7dq1pkWLFsbLy8v4+/ubLl26mH379jm1ufSxCdnS09NNq1atjK+vr9m0aZMx5u+fywkTJpgqVaoYT09PU6pUKdO8eXPzyiuvmPPnzzv1MbfHIVxa34kTJ8zQoUNNjRo1jI+PjwkICDBNmzY1H330UZ77jMLNZkwBnU0IoFCbPHmyYmJiCuyE5bw4ceKEQkNDNXHiRL3wwguuLqfQysrKUnBwsHr06KF58+a5uhzgpmAOFQBcwYIFC5SZmak+ffq4upRC49y5czlC9Pvvv69Tp07dkD89AxRUzKECgMt8/fXX2rdvn1566SV169aNPxuSD5s2bdKoUaPUs2dPlSxZUjt27NB7772nOnXqqGfPnq4uD7hpCFQAcJkpU6bo+++/V4sWLfTGG2+4upxCpUKFCgoPD9fs2bMdE+P79u2r6dOn5+shokBhwxwqAAAAi5hDBQAAYBGBCgAAwCLmUN0CWVlZOnbsmPz8/G7Znw4BAADWGGN05swZhYWFXfOP1BOoboFjx44pPDzc1WUAAIDrcPjwYZUtW/aqbQhUt0D2H8Q9fPiw/P39XVwNAADIi9TUVIWHhzv9YfsrIVDdAtmX+fz9/QlUAAAUMnmZrsOkdAAAAIsIVAAAABYRqAAAACwiUAEAAFhEoAIAALCIQAUAAGARj024hepMWiU3u7erywAAoEhJmN7J1SUwQgUAAGAVgQoAAMAiAhUAAIBFt3Wgio2Nlc1m0+nTp11dCgAAKMRuq0AVGRmp6OhoV5cBAACKmNsqUAEAANwMt02geuyxx7Rhwwa9/vrrstlsstlsSkhIkCRt375djRo1kre3t5o3b679+/c7fXbZsmW68847Vbx4cVWqVEkxMTG6ePGiC3oBAAAKotsmUL3++utq1qyZBg0apKSkJCUlJSk8PFyS9Nxzz+nVV1/Vtm3bVKxYMQ0YMMDxuW+//VZ9+/bVyJEjtW/fPr3zzjtasGCBXnrpJVd1BQAAFDC3TaAKCAiQp6envL29FRISopCQELm7u0uSXnrpJbVq1Uq1atXS+PHj9f333+vcuXOSpJiYGI0fP179+vVTpUqVdM899+jFF1/UO++8c8V9ZWRkKDU11ekFAACKLp6ULikiIsLxdWhoqCQpOTlZ5cqV065duxQXF+c0IpWZmalz584pPT1d3t45n3w+bdo0xcTE3PzCAQBAgUCgkuTh4eH42mazSZKysrIkSWlpaYqJiVGPHj1yfK548eK5bm/ChAkaPXq0431qaqrj8iIAACh6bqtA5enpqczMzHx95s4779T+/ftVpUqVPH/GbrfLbrfntzwAAFBI3VaBqkKFCtq8ebMSEhLk6+vrGIW6mokTJ6pz584qV66cHnjgAbm5uWnXrl3au3evpk6deguqBgAABd1tMyldksaOHSt3d3fVqlVLwcHBSkxMvOZnoqKi9MUXX2j16tVq3Lix7rrrLr322msqX778LagYAAAUBjZjjHF1EUVdamqqAgICFB79kdzsOSexAwCA65cwvdNN2W727++UlBT5+/tfte1tNUIFAABwMxCoAAAALCJQAQAAWHRb3eXnantjoq55DRYAABQ+jFABAABYRKACAACwiEAFAABgEYEKAADAIgIVAACARQQqAAAAiwhUAAAAFhGoAAAALCJQAQAAWESgAgAAsIhABQAAYBGBCgAAwCICFQAAgEUEKgAAAIsIVAAAABYRqAAAACwiUAEAAFhUzNUF3E7qTFolN7u3q8sAcImE6Z1cXQKAIoARKgAAAIsIVAAAABYRqAAAACwq1IEqMjJS0dHRri4DAADc5gp1oAIAACgICFRXcf78eVeXAAAACoEiE6gyMjI0duxYlSlTRj4+PmratKliY2Md60+ePKlevXqpTJky8vb2Vt26dbVo0SKnbURGRmrYsGGKjo5WqVKlFBUVpdjYWNlsNq1bt06NGjWSt7e3mjdvrv3799/iHgIAgIKqyASqYcOGaePGjVq8eLF2796tnj17qn379jpw4IAk6dy5c2rYsKFWrFihvXv3avDgwerTp4+2bNnitJ2FCxfK09NTcXFxevvttx3Ln3vuOb366qvatm2bihUrpgEDBtzS/gEAgILLZowxri7iekVGRqp+/foaPXq0KlWqpMTERIWFhTnWt2vXTk2aNNHLL7+c6+c7d+6sGjVq6JVXXnFsLzU1VTt27HC0iY2NVevWrbV27Vq1bdtWkvTll1+qU6dO+uuvv1S8ePEc283IyFBGRobjfWpqqsLDwxUe/REP9gQKGB7sCeBKUlNTFRAQoJSUFPn7+1+1bZF4UvqePXuUmZmpatWqOS3PyMhQyZIlJUmZmZl6+eWX9dFHH+no0aM6f/68MjIy5O3tHHAaNmyY6z4iIiIcX4eGhkqSkpOTVa5cuRxtp02bppiYGEt9AgAAhUeRCFRpaWlyd3fX9u3b5e7u7rTO19dXkjRz5ky9/vrrmjVrlurWrSsfHx9FR0fnmHju4+OT6z48PDwcX9tsNklSVlZWrm0nTJig0aNHO95nj1ABAICiqUgEqgYNGigzM1PJyclq2bJlrm3i4uLUtWtXPfroo5L+DkM///yzatWqdcPrsdvtstvtN3y7AACgYCoSk9KrVaum3r17q2/fvlq6dKkOHTqkLVu2aNq0aVqxYoUkqWrVqlqzZo2+//57xcfH64knntDvv//u4soBAEBRUCQClSTNnz9fffv21ZgxY1S9enV169ZNW7dudcxxev7553XnnXcqKipKkZGRCgkJUbdu3VxbNAAAKBIK9V1+hUX2XQLc5QcUPNzlB+BK8nOXX5EZoQIAAHAVAhUAAIBFBCoAAACLisRjEwqLvTFR17wGCwAACh9GqAAAACwiUAEAAFhEoAIAALCIQAUAAGARgQoAAMAiAhUAAIBFBCoAAACLCFQAAAAWEagAAAAsIlABAABYRKACAACwiEAFAABgEYEKAADAIgIVAACARQQqAAAAiwhUAAAAFhGoAAAALCrm6gJuJ3UmrZKb3dvVZQBFWsL0Tq4uAcBtiBEqAAAAiwhUAAAAFhGoAAAALCJQSYqMjFR0dHSe2y9YsECBgYE3rR4AAFC4EKgAAAAsIlABAABYVKADVWRkpIYPH67o6GiVKFFCpUuX1rx583T27Fn1799ffn5+qlKlilauXOn4zIYNG9SkSRPZ7XaFhoZq/PjxunjxomP92bNn1bdvX/n6+io0NFSvvvpqjv1mZGRo7NixKlOmjHx8fNS0aVPFxsbeii4DAIBCqEAHKklauHChSpUqpS1btmj48OEaMmSIevbsqebNm2vHjh2699571adPH6Wnp+vo0aPq2LGjGjdurF27dmnOnDl67733NHXqVMf2xo0bpw0bNmjZsmVavXq1YmNjtWPHDqd9Dhs2TBs3btTixYu1e/du9ezZU+3bt9eBAwdudfcBAEAhYDPGGFcXcSWRkZHKzMzUt99+K0nKzMxUQECAevTooffff1+SdPz4cYWGhmrjxo1avny5lixZovj4eNlsNknSW2+9pWeeeUYpKSlKT09XyZIl9d///lc9e/aUJJ06dUply5bV4MGDNWvWLCUmJqpSpUpKTExUWFiYo5Z27dqpSZMmevnll7VgwQJFR0fr9OnTudadkZGhjIwMx/vU1FSFh4crPPojHuwJ3GQ82BPAjZKamqqAgAClpKTI39//qm0L/JPSIyIiHF+7u7urZMmSqlu3rmNZ6dKlJUnJycmKj49Xs2bNHGFKklq0aKG0tDQdOXJEf/75p86fP6+mTZs61gcFBal69eqO93v27FFmZqaqVavmVEdGRoZKliyZp5qnTZummJiY/HUUAAAUWgU+UHl4eDi9t9lsTsuyw1NWVtYN2V9aWprc3d21fft2ubu7O63z9fXN0zYmTJig0aNHO95nj1ABAICiqcAHqvyoWbOmlixZImOMI2jFxcXJz89PZcuWVVBQkDw8PLR582aVK1dOkvTnn3/q559/VqtWrSRJDRo0UGZmppKTk9WyZcvrqsNut8tut9+YTgEAgAKvwE9Kz4+nnnpKhw8f1vDhw/XTTz9p2bJlmjRpkkaPHi03Nzf5+vpq4MCBGjdunL7++mvt3btXjz32mNzc/t9hqFatmnr37q2+fftq6dKlOnTokLZs2aJp06ZpxYoVLuwdAAAoqIrUCFWZMmX05Zdfaty4capXr56CgoI0cOBAPf/88442M2fOVFpamrp06SI/Pz+NGTNGKSkpTtuZP3++pk6dqjFjxujo0aMqVaqU7rrrLnXu3PlWdwkAABQCBfouv6Ii+y4B7vIDbj7u8gNwo+TnLr8idckPAADAFQhUAAAAFhGoAAAALCpSk9ILur0xUde8BgsAAAofRqgAAAAsIlABAABYRKACAACwiEAFAABgEYEKAADAIgIVAACARQQqAAAAiwhUAAAAFhGoAAAALCJQAQAAWESgAgAAsIhABQAAYBGBCgAAwCICFQAAgEUEKgAAAIsIVAAAABYRqAAAACwq5uoCbid1Jq2Sm93b1WUARVrC9E6uLgHAbYgRKgAAAIsIVAAAABYRqAAAACwiUF2HuLg41a1bVx4eHurWrZurywEAAC7GpPTrMHr0aNWvX18rV66Ur6+vq8sBAAAuxgjVdTh48KDatGmjsmXLKjAw0NXlAAAAFyNQ5SIjI0MjRozQHXfcoeLFi+vuu+/W1q1blZCQIJvNppMnT2rAgAGy2WxasGCBq8sFAAAuRqDKxdNPP60lS5Zo4cKF2rFjh6pUqaKoqCj5+fkpKSlJ/v7+mjVrlpKSkvTQQw/l+HxGRoZSU1OdXgAAoOgiUF3m7NmzmjNnjmbOnKkOHTqoVq1amjdvnry8vPTvf/9bISEhstlsCggIUEhIiLy8vHJsY9q0aQoICHC8wsPDXdATAABwqxCoLnPw4EFduHBBLVq0cCzz8PBQkyZNFB8fn6dtTJgwQSkpKY7X4cOHb1a5AACgAOAuv5vAbrfLbre7ugwAAHCLMEJ1mcqVK8vT01NxcXGOZRcuXNDWrVtVq1YtF1YGAAAKKkaoLuPj46MhQ4Zo3LhxCgoKUrly5TRjxgylp6dr4MCBri4PAAAUQASqXEyfPl1ZWVnq06ePzpw5o0aNGmnVqlUqUaKEq0sDAAAFEIEqF8WLF9fs2bM1e/bsXNefPn361hYEAAAKNOZQAQAAWESgAgAAsIhABQAAYBFzqG6hvTFR8vf3d3UZAADgBmOECgAAwCICFQAAgEUEKgAAAIsIVAAAABYRqAAAACwiUAEAAFhEoAIAALCIQAUAAGARgQoAAMAiAhUAAIBFBCoAAACLCFQAAAAWEagAAAAsIlABAABYRKACAACwiEAFAABgEYEKAADAomKuLuB2UmfSKrnZvV1dBlCkJUzv5OoSANyGGKECAACwiEAFAABgEYEKAADAIgLVdViwYIECAwNdXQYAACggCFQAAAAWEagAAAAsKvKB6quvvtLdd9+twMBAlSxZUp07d9bBgwclSQkJCbLZbFq6dKlat24tb29v1atXTxs3bnTaxoIFC1SuXDl5e3ure/fuOnnypCu6AgAACqgiH6jOnj2r0aNHa9u2bVq3bp3c3NzUvXt3ZWVlOdo899xzGjt2rHbu3Klq1aqpV69eunjxoiRp8+bNGjhwoIYNG6adO3eqdevWmjp16lX3mZGRodTUVKcXAAAoumzGGOPqIm6lEydOKDg4WHv27JGvr68qVqyod999VwMHDpQk7du3T7Vr11Z8fLxq1KihRx55RCkpKVqxYoVjGw8//LC++uornT59Otd9TJ48WTExMTmWh0d/xIM9gZuMB3sCuFFSU1MVEBCglJQU+fv7X7XtdT0p/fTp09qyZYuSk5OdRnokqW/fvtezyZvmwIEDmjhxojZv3qwTJ0446k1MTFStWrUkSREREY72oaGhkqTk5GTVqFFD8fHx6t69u9M2mzVrpq+++uqK+5wwYYJGjx7teJ+amqrw8PAb1icAAFCw5DtQLV++XL1791ZaWpr8/f1ls9kc62w2W4ELVF26dFH58uU1b948hYWFKSsrS3Xq1NH58+cdbTw8PBxfZ/fn8qCYH3a7XXa7/fqLBgAAhUq+51CNGTNGAwYMUFpamk6fPq0///zT8Tp16tTNqPG6nTx5Uvv379fzzz+vtm3bqmbNmvrzzz/ztY2aNWtq8+bNTss2bdp0I8sEAACFXL5HqI4ePaoRI0bI27vgzwUqUaKESpYsqblz5yo0NFSJiYkaP358vrYxYsQItWjRQq+88oq6du2qVatWXfVyHwAAuP3ke4QqKipK27Ztuxm13HBubm5avHixtm/frjp16mjUqFGaOXNmvrZx1113ad68eXr99ddVr149rV69Ws8///xNqhgAABRG+b7L77333tOUKVPUv39/1a1b12n+kSTdd999N7TAoiD7LgHu8gNuPu7yA3Cj3NS7/AYNGiRJmjJlSo51NptNmZmZ+d0kAABAoZbvQGXl7jcAAICiyNKT0s+dO3ej6gAAACi08j1ClZmZqZdffllvv/22fv/9d/3888+qVKmSXnjhBVWoUMHxxHHktDcm6prXYAEAQOGT7xGql156SQsWLNCMGTPk6enpWF6nTh29++67N7Q4AACAwiDfger999/X3Llz1bt3b7m7uzuW16tXTz/99NMNLQ4AAKAwyHegOnr0qKpUqZJjeVZWli5cuHBDigIAAChM8h2oatWqpW+//TbH8k8++UQNGjS4IUUBAAAUJvmelD5x4kT169dPR48eVVZWlpYuXar9+/fr/fff1xdffHEzagQAACjQ8j1C1bVrVy1fvlxr166Vj4+PJk6cqPj4eC1fvlz33HPPzagRAACgQMv3CNWRI0fUsmVLrVmzJse6TZs26a677rohhQEAABQW+R6huvfee3Xq1Kkcy+Pi4tS+ffsbUhQAAEBhku9Addddd+nee+/VmTNnHMu++eYbdezYUZMmTbqhxQEAABQG+Q5U7777rsqVK6cuXbooIyND69evV6dOnTRlyhSNGjXqZtQIAABQoOU7ULm5uWnx4sXy8PBQmzZtdN9992natGkaOXLkzagPAACgwLMZY8y1Gu3evTvHsjNnzqhXr17q1KmThgwZ4lgeERFxYyssAlJTUxUQEKCUlBT+lh8AAIVEfn5/5ylQubm5yWaz6dKml77P/tpmsykzM9Ni+UUPgQoAgMInP7+/8/TYhEOHDt2QwgAAAIqiPAWq8uXL3+w6AAAACq18P9hTkg4ePKhZs2YpPj5e0t9/32/kyJGqXLnyDS0OAACgMMj3XX6rVq1SrVq1tGXLFkVERCgiIkKbN29W7dq1c316OgAAQFGXp0npl2rQoIGioqI0ffp0p+Xjx4/X6tWrtWPHjhtaYFGQPaktPPojudm9XV0OUOgkTO/k6hIA3IbyMyk93yNU8fHxGjhwYI7lAwYM0L59+/K7OQAAgEIv34EqODhYO3fuzLF8586duuOOO25ETQAAAIVKnielT5kyRWPHjtWgQYM0ePBg/frrr2revLmkv/8w8j//+U+NHj36phUKAABQUOV5DpW7u7uSkpIUHBysWbNm6dVXX9WxY8ckSWFhYRo3bpxGjBghm812Uwu+lSIjI1W/fn3NmjXL0naYQwVYwxwqAK5wwx/sKcnpqeijRo3SqFGjdObMGUmSn5+fhXILrqVLl8rDw8PVZQAAgAIuX8+hunz0qagGqWxBQUGuLgEAABQC+ZqUXq1aNQUFBV31VZRERkYqOjpakvTWW2+patWqKl68uEqXLq0HHnjAtcUBAIACI18jVDExMQoICLhZtRRY27Zt04gRI/Sf//xHzZs316lTp/Ttt99esX1GRoYyMjIc71NTU29FmQAAwEXyFagefvjh2/LRCImJifLx8VHnzp3l5+en8uXLq0GDBldsP23aNMXExNzCCgEAgCvl+ZJfUbp7L7/uuecelS9fXpUqVVKfPn30wQcfKD09/YrtJ0yYoJSUFMfr8OHDt7BaAABwq+U5UOXzL9QUKX5+ftqxY4cWLVqk0NBQTZw4UfXq1dPp06dzbW+32+Xv7+/0AgAARVeeA1VWVtZtebkvW7FixdSuXTvNmDFDu3fvVkJCgr7++mtXlwUAAAqAfM2hul198cUX+vXXX/WPf/xDJUqU0JdffqmsrCxVr17d1aUBAIACgECVB4GBgVq6dKkmT56sc+fOqWrVqlq0aJFq167t6tIAAEABQKC6itjY2Fy/BgAAuFS+HuwJAACAnAhUAAAAFhGoAAAALGIO1S20NyaKZ1IBAFAEMUIFAABgEYEKAADAIgIVAACARQQqAAAAiwhUAAAAFhGoAAAALCJQAQAAWESgAgAAsIhABQAAYBGBCgAAwCICFQAAgEUEKgAAAIsIVAAAABYRqAAAACwiUAEAAFhEoAIAALCIQAUAAGBRMVcXcDupM2mV3Ozeri4DyJOE6Z1cXQIAFBqMUAEAAFhEoAIAALCIQAUAAGDRbReobDabPvvssyuuj42Nlc1m0+nTp29ZTQAAoHC77QLVtTRv3lxJSUkKCAhwdSkAAKCQ4C6/y3h6eiokJMTVZQAAgELEpSNUkZGRGj58uKKjo1WiRAmVLl1a8+bN09mzZ9W/f3/5+fmpSpUqWrlypSQpMzNTAwcOVMWKFeXl5aXq1avr9ddfz7Hdf//736pdu7bsdrtCQ0M1bNgwp/UnTpxQ9+7d5e3trapVq+rzzz93rLv8kt+CBQsUGBioVatWqWbNmvL19VX79u2VlJR08w4MAAAoVFx+yW/hwoUqVaqUtmzZouHDh2vIkCHq2bOnmjdvrh07dujee+9Vnz59lJ6erqysLJUtW1Yff/yx9u3bp4kTJ+rZZ5/VRx995NjenDlzNHToUA0ePFh79uzR559/ripVqjjtMyYmRg8++KB2796tjh07qnfv3jp16tQVa0xPT9crr7yi//znP/rmm2+UmJiosWPHXrF9RkaGUlNTnV4AAKDoshljjKt2HhkZqczMTH377beS/h6BCggIUI8ePfT+++9Lko4fP67Q0FBt3LhRd911V45tDBs2TMePH9cnn3wiSSpTpoz69++vqVOn5rpPm82m559/Xi+++KIk6ezZs/L19dXKlSvVvn17xcbGqnXr1vrzzz8VGBioBQsWqH///vrll19UuXJlSdJbb72lKVOm6Pjx47nuY/LkyYqJicmxPDz6Ix7siUKDB3sCuN2lpqYqICBAKSkp8vf3v2pbl49QRUREOL52d3dXyZIlVbduXcey0qVLS5KSk5MlSW+++aYaNmyo4OBg+fr6au7cuUpMTHS0OXbsmNq2bZvnffr4+Mjf39+x/dx4e3s7wpQkhYaGXrX9hAkTlJKS4ngdPnz4qvUAAIDCzeWT0j08PJze22w2p2U2m02SlJWVpcWLF2vs2LF69dVX1axZM/n5+WnmzJnavHmzJMnLy+u695mVlZWv9lcb2LPb7bLb7XmqBQAAFH4uD1T5ERcXp+bNm+upp55yLDt48KDjaz8/P1WoUEHr1q1T69atXVEiAAC4DRWqQFW1alW9//77WrVqlSpWrKj//Oc/2rp1qypWrOhoM3nyZD355JO644471KFDB505c0ZxcXEaPny4CysHAABFmcvnUOXHE088oR49euihhx5S06ZNdfLkSafRKknq16+fZs2apbfeeku1a9dW586ddeDAARdVDAAAbgcuvcvvdpF9lwB3+aEw4S4/ALe7QnWXHwAAQGFHoAIAALCIQAUAAGBRobrLr7DbGxN1zWuwAACg8GGECgAAwCICFQAAgEUEKgAAAIsIVAAAABYRqAAAACwiUAEAAFhEoAIAALCIQAUAAGARgQoAAMAiAhUAAIBFBCoAAACLCFQAAAAWEagAAAAsIlABAABYRKACAACwiEAFAABgEYEKAADAomKuLuB2UmfSKrnZvV1dBpAnCdM7uboEACg0GKECAACwiEAFAABgEYEKAADAIgIVAACARQSqy0RGRio6OtrVZQAAgEKEQAUAAGARgeoSjz32mDZs2KDXX39dNptNNptNCQkJ2rBhg5o0aSK73a7Q0FCNHz9eFy9edHW5AACggCBQXeL1119Xs2bNNGjQICUlJSkpKUkeHh7q2LGjGjdurF27dmnOnDl67733NHXq1CtuJyMjQ6mpqU4vAABQdPFgz0sEBATI09NT3t7eCgkJkSQ999xzCg8P17/+9S/ZbDbVqFFDx44d0zPPPKOJEyfKzS1nJp02bZpiYmJudfkAAMBFGKG6hvj4eDVr1kw2m82xrEWLFkpLS9ORI0dy/cyECROUkpLieB0+fPhWlQsAAFyAEaqbwG63y263u7oMAABwizBCdRlPT09lZmY63tesWVMbN26UMcaxLC4uTn5+fipbtqwrSgQAAAUMgeoyFSpU0ObNm5WQkKATJ07oqaee0uHDhzV8+HD99NNPWrZsmSZNmqTRo0fnOn8KAADcfkgElxk7dqzc3d1Vq1YtBQcH68KFC/ryyy+1ZcsW1atXT08++aQGDhyo559/3tWlAgCAAoI5VJepVq2aNm7c6LSsQoUK2rJli4sqAgAABR0jVAAAABYRqAAAACwiUAEAAFjEHKpbaG9MlPz9/V1dBgAAuMEYoQIAALCIQAUAAGARgQoAAMAiAhUAAIBFBCoAAACLCFQAAAAWEagAAAAsIlABAABYRKACAACwiEAFAABgEYEKAADAIgIVAACARQQqAAAAiwhUAAAAFhGoAAAALCJQAQAAWESgAgAAsKiYqwu4ndSZtEpudm9Xl4EiJGF6J1eXAAAQI1QAAACWEagAAAAsIlABAABYRKACAACwiEAFAABgEYHKAmOMLl686OoyAACAixGoLpORkaERI0bojjvuUPHixXX33Xdr69atkqTY2FjZbDatXLlSDRs2lN1u13fffefiigEAgKsRqC7z9NNPa8mSJVq4cKF27NihKlWqKCoqSqdOnXK0GT9+vKZPn674+HhFRETk2EZGRoZSU1OdXgAAoOgiUF3i7NmzmjNnjmbOnKkOHTqoVq1amjdvnry8vPTee+852k2ZMkX33HOPKleurKCgoBzbmTZtmgICAhyv8PDwW9kNAABwixGoLnHw4EFduHBBLVq0cCzz8PBQkyZNFB8f71jWqFGjq25nwoQJSklJcbwOHz5802oGAACux5+euQ4+Pj5XXW+322W3229RNQAAwNUYobpE5cqV5enpqbi4OMeyCxcuaOvWrapVq5YLKwMAAAUZI1SX8PHx0ZAhQzRu3DgFBQWpXLlymjFjhtLT0zVw4EDt2rXL1SUCAIACiEB1menTpysrK0t9+vTRmTNn1KhRI61atUolSpRwdWkAAKCAshljjKuLKOpSU1P/vtsv+iO52b1dXQ6KkITpnVxdAgAUWdm/v1NSUuTv73/VtsyhAgAAsIhABQAAYBGBCgAAwCImpd9Ce2OirnkNFgAAFD6MUAEAAFhEoAIAALCIQAUAAGARgQoAAMAiAhUAAIBFBCoAAACLCFQAAAAWEagAAAAsIlABAABYRKACAACwiEAFAABgEYEKAADAIgIVAACARQQqAAAAiwhUAAAAFhGoAAAALCJQAQAAWFTM1QXcTupMWiU3u7ery0ARkjC9k6tLAACIESoAAADLCFQAAAAWEagAAAAsuu0CVYUKFTRr1izHe5vNps8++8xl9QAAgMLvtgtUl0tKSlKHDh3y1Hby5MmqX7/+zS0IAAAUOoXyLr/z58/L09PzhmwrJCTkhmwHAADcvgrECFVkZKSGDRumYcOGKSAgQKVKldILL7wgY4ykvy/Tvfjii+rbt6/8/f01ePBgSdJ3332nli1bysvLS+Hh4RoxYoTOnj3r2G5ycrK6dOkiLy8vVaxYUR988EGOfV9+ye/IkSPq1auXgoKC5OPjo0aNGmnz5s1asGCBYmJitGvXLtlsNtlsNi1YsOCmHhcAAFA4FJgRqoULF2rgwIHasmWLtm3bpsGDB6tcuXIaNGiQJOmVV17RxIkTNWnSJEnSwYMH1b59e02dOlX//ve/9ccffzhC2fz58yVJjz32mI4dO6b169fLw8NDI0aMUHJy8hVrSEtLU6tWrVSmTBl9/vnnCgkJ0Y4dO5SVlaWHHnpIe/fu1VdffaW1a9dKkgICAnLdTkZGhjIyMhzvU1NTb8gxAgAABVOBCVTh4eF67bXXZLPZVL16de3Zs0evvfaaI1C1adNGY8aMcbR//PHH1bt3b0VHR0uSqlatqtmzZ6tVq1aaM2eOEhMTtXLlSm3ZskWNGzeWJL333nuqWbPmFWv48MMP9ccff2jr1q0KCgqSJFWpUsWx3tfXV8WKFbvmZcJp06YpJibmuo4DAAAofArEJT9Juuuuu2Sz2RzvmzVrpgMHDigzM1OS1KhRI6f2u3bt0oIFC+Tr6+t4RUVFKSsrS4cOHVJ8fLyKFSumhg0bOj5To0YNBQYGXrGGnTt3qkGDBo4wdb0mTJiglJQUx+vw4cOWtgcAAAq2AjNCdS0+Pj5O79PS0vTEE09oxIgROdqWK1dOP//8c7734eXldd31Xcput8tut9+QbQEAgIKvwASqzZs3O73ftGmTqlatKnd391zb33nnndq3b5/TJblL1ahRQxcvXtT27dsdl/z279+v06dPX7GGiIgIvfvuuzp16lSuo1Senp6OETMAAIBsBeaSX2JiokaPHq39+/dr0aJFeuONNzRy5Mgrtn/mmWf0/fffa9iwYdq5c6cOHDigZcuWadiwYZKk6tWrq3379nriiSe0efNmbd++XY8//vhVR6F69eqlkJAQdevWTXFxcfr111+1ZMkSbdy4UdLfdxseOnRIO3fu1IkTJ5wmngMAgNtXgQlUffv21V9//aUmTZpo6NChGjlypOPxCLmJiIjQhg0b9PPPP6tly5Zq0KCBJk6cqLCwMEeb+fPnKywsTK1atVKPHj00ePBg3XHHHVfcpqenp1avXq077rhDHTt2VN26dTV9+nTHKNn999+v9u3bq3Xr1goODtaiRYtu3AEAAACFls1kP+zJhSIjI1W/fn2nPwlTlKSmpiogIEDh0R/Jze7t6nJQhCRM7+TqEgCgyMr+/Z2SkiJ/f/+rti0wI1QAAACFFYEKAADAogJxya+oy8+QIQAAKBi45AcAAHALEagAAAAsIlABAABYRKACAACwiEAFAABgEYEKAADAIgIVAACARQQqAAAAi4q5uoDbQfazU1NTU11cCQAAyKvs39t5eQY6geoWOHnypCQpPDzcxZUAAID8OnPmjAICAq7ahkB1CwQFBUmSEhMTr3lCCrPU1FSFh4fr8OHDRfpP7NwO/bwd+ijdHv28Hfoo0c+ipCD10RijM2fOKCws7JptCVS3gJvb31PVAgICXP7NcSv4+/vTzyLiduijdHv083boo0Q/i5KC0se8DoQwKR0AAMAiAhUAAIBFBKpbwG63a9KkSbLb7a4u5aain0XH7dBH6fbo5+3QR4l+FiWFtY82k5d7AQEAAHBFjFABAABYRKACAACwiEAFAABgEYEKAADAIgLVLfDmm2+qQoUKKl68uJo2baotW7a4uqQbavLkybLZbE6vGjVquLosS7755ht16dJFYWFhstls+uyzz5zWG2M0ceJEhYaGysvLS+3atdOBAwdcU6wF1+rnY489luPctm/f3jXFXqdp06apcePG8vPz0x133KFu3bpp//79Tm3OnTunoUOHqmTJkvL19dX999+v33//3UUVX5+89DMyMjLH+XzyySddVHH+zZkzRxEREY4HPjZr1kwrV650rC8K51G6dj8L+3nMzfTp02Wz2RQdHe1YVtjOJ4HqJvvf//6n0aNHa9KkSdqxY4fq1aunqKgoJScnu7q0G6p27dpKSkpyvL777jtXl2TJ2bNnVa9ePb355pu5rp8xY4Zmz56tt99+W5s3b5aPj4+ioqJ07ty5W1ypNdfqpyS1b9/e6dwuWrToFlZo3YYNGzR06FBt2rRJa9as0YULF3Tvvffq7NmzjjajRo3S8uXL9fHHH2vDhg06duyYevTo4cKq8y8v/ZSkQYMGOZ3PGTNmuKji/CtbtqymT5+u7du3a9u2bWrTpo26du2qH3/8UVLROI/StfspFe7zeLmtW7fqnXfeUUREhNPyQnc+DW6qJk2amKFDhzreZ2ZmmrCwMDNt2jQXVnVjTZo0ydSrV8/VZdw0ksynn37qeJ+VlWVCQkLMzJkzHctOnz5t7Ha7WbRokQsqvDEu76cxxvTr18907drVJfXcLMnJyUaS2bBhgzHm73Pn4eFhPv74Y0eb+Ph4I8ls3LjRVWVadnk/jTGmVatWZuTIka4r6iYoUaKEeffdd4vsecyW3U9jitZ5PHPmjKlatapZs2aNU78K4/lkhOomOn/+vLZv36527do5lrm5ualdu3bauHGjCyu78Q4cOKCwsDBVqlRJvXv3VmJioqtLumkOHTqk48ePO53XgIAANW3atMidV0mKjY3VHXfcoerVq2vIkCE6efKkq0uyJCUlRdL/+6Pl27dv14ULF5zOZ40aNVSuXLlCfT4v72e2Dz74QKVKlVKdOnU0YcIEpaenu6I8yzIzM7V48WKdPXtWzZo1K7Ln8fJ+Zisq53Ho0KHq1KmT03mTCufPJX8c+SY6ceKEMjMzVbp0aaflpUuX1k8//eSiqm68pk2basGCBapevbqSkpIUExOjli1bau/evfLz83N1eTfc8ePHJSnX85q9rqho3769evTooYoVK+rgwYN69tln1aFDB23cuFHu7u6uLi/fsrKyFB0drRYtWqhOnTqS/j6fnp6eCgwMdGpbmM9nbv2UpEceeUTly5dXWFiYdu/erWeeeUb79+/X0qVLXVht/uzZs0fNmjXTuXPn5Ovrq08//VS1atXSzp07i9R5vFI/paJxHiVp8eLF2rFjh7Zu3ZpjXWH8uSRQwbIOHTo4vo6IiFDTpk1Vvnx5ffTRRxo4cKALK4NVDz/8sOPrunXrKiIiQpUrV1ZsbKzatm3rwsquz9ChQ7V3795CP8fvWq7Uz8GDBzu+rlu3rkJDQ9W2bVsdPHhQlStXvtVlXpfq1atr586dSklJ0SeffKJ+/fppw4YNri7rhrtSP2vVqlUkzuPhw4c1cuRIrVmzRsWLF3d1OTcEl/xuolKlSsnd3T3HXQm///67QkJCXFTVzRcYGKhq1arpl19+cXUpN0X2ubvdzqskVapUSaVKlSqU53bYsGH64osvtH79epUtW9axPCQkROfPn9fp06ed2hfW83mlfuamadOmklSozqenp6eqVKmihg0batq0aapXr55ef/31Incer9TP3BTG87h9+3YlJyfrzjvvVLFixVSsWDFt2LBBs2fPVrFixVS6dOlCdz4JVDeRp6enGjZsqHXr1jmWZWVlad26dU7XwouatLQ0HTx4UKGhoa4u5aaoWLGiQkJCnM5ramqqNm/eXKTPqyQdOXJEJ0+eLFTn1hijYcOG6dNPP9XXX3+tihUrOq1v2LChPDw8nM7n/v37lZiYWKjO57X6mZudO3dKUqE6n5fLyspSRkZGkTmPV5Ldz9wUxvPYtm1b7dmzRzt37nS8GjVqpN69ezu+LnTn09Wz4ou6xYsXG7vdbhYsWGD27dtnBg8ebAIDA83x48ddXdoNM2bMGBMbG2sOHTpk4uLiTLt27UypUqVMcnKyq0u7bmfOnDE//PCD+eGHH4wk83//93/mhx9+ML/99psxxpjp06ebwMBAs2zZMrN7927TtWtXU7FiRfPXX3+5uPL8uVo/z5w5Y8aOHWs2btxoDh06ZNauXWvuvPNOU7VqVXPu3DlXl55nQ4YMMQEBASY2NtYkJSU5Xunp6Y42Tz75pClXrpz5+uuvzbZt20yzZs1Ms2bNXFh1/l2rn7/88ouZMmWK2bZtmzl06JBZtmyZqVSpkvnHP/7h4srzbvz48WbDhg3m0KFDZvfu3Wb8+PHGZrOZ1atXG2OKxnk05ur9LArn8Uouv3uxsJ1PAtUt8MYbb5hy5coZT09P06RJE7Np0yZXl3RDPfTQQyY0NNR4enqaMmXKmIceesj88ssvri7LkvXr1xtJOV79+vUzxvz96IQXXnjBlC5d2tjtdtO2bVuzf/9+1xZ9Ha7Wz/T0dHPvvfea4OBg4+HhYcqXL28GDRpU6P4zkFv/JJn58+c72vz111/mqaeeMiVKlDDe3t6me/fuJikpyXVFX4dr9TMxMdH84x//MEFBQcZut5sqVaqYcePGmZSUFNcWng8DBgww5cuXN56eniY4ONi0bdvWEaaMKRrn0Zir97MonMcruTxQFbbzaTPGmFs3HgYAAFD0MIcKAADAIgIVAACARQQqAAAAiwhUAAAAFhGoAAAALCJQAQAAWESgAgAAsIhABQD/v8mTJ6t06dKy2Wz67LPPXF0OgEKEQAUg344fP67hw4erUqVKstvtCg8PV5cuXZz+7tatcqPCT3x8vGJiYvTOO+8oKSlJHTp0sF4cgNtGMVcXAKBwSUhIUIsWLRQYGKiZM2eqbt26unDhglatWqWhQ4fqp59+cnWJ1+XgwYOSpK5du8pms7m4mrw7f/68PD09nZYZY5SZmalixfgnHrhVGKECkC9PPfWUbDabtmzZovvvv1/VqlVT7dq1NXr0aG3atMnRLjExUV27dpWvr6/8/f314IMP6vfff3esf+yxx9StWzenbUdHRysyMtLxPjIyUiNGjNDTTz+toKAghYSEaPLkyY71FSpUkCR1795dNpvN8T43e/bsUZs2beTl5aWSJUtq8ODBSktLk/T3pb4uXbpIktzc3K4aqDZs2KAmTZrIbrcrNDRU48eP18WLFx3rs7KyNGPGDFWpUkV2u13lypXTSy+95Fh/5MgR9erVS0FBQfLx8VGjRo20efPmfB2TYcOGKTo6WqVKlVJUVJRiY2Nls9m0cuVKNWzYUHa7Xd99952ysrI0bdo0VaxYUV5eXqpXr54++eQTx7ayP7du3To1atRI3t7eat68ufbv3+9Uw/Lly9W4cWMVL15cpUqVUvfu3R3rMjIyNHbsWJUpU0Y+Pj5q2rSpYmNjr3j8gKKKQAUgz06dOqWvvvpKQ4cOlY+PT471gYGBkv4OFV27dtWpU6e0YcMGrVmzRr/++qseeuihfO9z4cKF8vHx0ebNmzVjxgxNmTJFa9askSRt3bpVkjR//nwlJSU53l/u7NmzioqKUokSJbR161Z9/PHHWrt2rYYNGyZJGjt2rObPny9JSkpKUlJSUq7bOXr0qDp27KjGjRtr165dmjNnjt577z1NnTrV0WbChAmaPn26XnjhBe3bt08ffvihSpcuLUlKS0tTq1atdPToUX3++efatWuXnn76aWVlZeX7mHh6eiouLk5vv/22Y/n48eM1ffp0xcfHKyIiQtOmTdP777+vt99+Wz/++KNGjRqlRx99VBs2bHDa3nPPPadXX31V27ZtU7FixTRgwADHuhUrVqh79+7q2LGjfvjhB61bt05NmjRxrB82bJg2btyoxYsXa/fu3erZs6fat2+vAwcO5KtPQKHn4j/ODKAQ2bx5s5Fkli5detV2q1evNu7u7iYxMdGx7McffzSSzJYtW4wxxvTr18907drV6XMjR440rVq1crxv1aqVufvuu53aNG7c2DzzzDOO95LMp59+etV65s6da0qUKGHS0tIcy1asWGHc3NzM8ePHjTHGfPrpp+Za/yQ+++yzpnr16iYrK8ux7M033zS+vr4mMzPTpKamGrvdbubNm5fr59955x3j5+dnTp48mev6vB6TBg0aOLVZv369kWQ+++wzx7Jz584Zb29v8/333zu1HThwoOnVq5fT59auXetYv2LFCiPJ/PXXX8YYY5o1a2Z69+6da72//fabcXd3N0ePHnVa3rZtWzNhwoRcPwMUVVxgB5Bnxpg8tYuPj1d4eLjCw8Mdy2rVqqXAwEDFx8ercePGed5nRESE0/vQ0FAlJyfn+fPZ9dSrV89pVK1FixbKysrS/v37HSNIedlOs2bNnC4JtmjRQmlpaTpy5IiOHz+ujIwMtW3bNtfP79y5Uw0aNFBQUFC+6r9cw4YNc13eqFEjx9e//PKL0tPTdc899zi1OX/+vBo0aOC07NJjHBoaKklKTk5WuXLltHPnTg0aNCjX/e3Zs0eZmZmqVq2a0/KMjAyVLFky7x0CigACFYA8q1q1qmw22w2ZeO7m5pYjoF24cCFHOw8PD6f3Npst35fIbhUvLy9L6/N6THK73Hr58uz5YStWrFCZMmWc2tntdqf3lx7j7LCYfYyvVnNaWprc3d21fft2ubu7O63z9fW94ueAoog5VADyLCgoSFFRUXrzzTd19uzZHOtPnz4tSapZs6YOHz6sw4cPO9bt27dPp0+fVq1atSRJwcHBOeYq7dy5M981eXh4KDMz86ptatasqV27djnVHBcXJzc3N1WvXj3P+6pZs6Y2btzoFHri4uLk5+ensmXLqmrVqvLy8rri4yMiIiK0c+dOnTp1Ktf1N+qYSH+PCNrtdiUmJqpKlSpOr0tHDq8lIiLiiv1p0KCBMjMzlZycnGMfISEh11U3UFgRqADky5tvvqnMzEw1adJES5Ys0YEDBxQfH6/Zs2erWbNmkqR27dqpbt266t27t3bs2KEtW7aob9++atWqleOyVJs2bbRt2za9//77OnDggCZNmqS9e/fmu54KFSpo3bp1On78uP78889c2/Tu3VvFixdXv379tHfvXq1fv17Dhw9Xnz598ny5T/r7DsfDhw9r+PDh+umnn7Rs2TJNmjRJo0ePlpubm4oXL65nnnlGTz/9tN5//30dPHhQmzZt0nvvvSdJ6tWrl0JCQtStWzfFxcXp119/1ZIlS7Rx48Ybekwkyc/PT2PHjtWoUaO0cOFCHTx4UDt27NAbb7yhhQsX5nk7kyZN0qJFizRp0iTFx8drz549+uc//ylJqlatmnr37q2+fftq6dKlOnTokLZs2aJp06ZpxYoV11U3UGi5dgoXgMLo2LFjZujQoaZ8+fLG09PTlClTxtx3331m/fr1jja//fabue+++4yPj4/x8/MzPXv2dEwAzzZx4kRTunRpExAQYEaNGmWGDRuWYwL2yJEjnT7TtWtX069fP8f7zz//3FSpUsUUK1bMlC9f/oo1796927Ru3doUL17cBAUFmUGDBpkzZ8441udlUroxxsTGxprGjRsbT09PExISYp555hlz4cIFx/rMzEwzdepUU758eePh4WHKlStnXn75Zcf6hIQEc//99xt/f3/j7e1tGjVqZDZv3mzpmGRPLv/zzz+dlmdlZZlZs2aZ6tWrGw8PDxMcHGyioqLMhg0brvi5H374wUgyhw4dcixbsmSJqV+/vvH09DSlSpUyPXr0cKw7f/68mThxoqlQoYLx8PAwoaGhpnv37mb37t3XPJZAUWIzJo+zTAEAAJArLvkBAABYRKACAACwiEAFAABgEYEKAADAIgIVAACARQQqAAAAiwhUAAAAFhGoAAAALCJQAQAAWESgAgAAsIhABQAAYBGBCgAAwKL/DwJgo4BowFJlAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "import matplotlib.pyplot as plt\n", "count_df.T[0].sort_values().tail(10).plot(kind='barh')\n", "plt.xlabel('Count of occurrence')\n", "plt.ylabel('Token')\n", "plt.title('Most Frequently Occuring Tokens')\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": { "id": "LJcUg2k-fdnC" }, "source": [ "We can see that many of the most frequently occuring tokens are [stop words](https://en.wikipedia.org/wiki/Stop_word). We could have dealt with these as part of preprocessing, however, fortunately for us basic stopword removal is built into the vectorizers in scikit-learn and so this can be done concurrently.\n", "\n", "So we will take a step back here and re-vectorize our data, while removing stopwords, then replot the result:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 472 }, "executionInfo": { "elapsed": 309, "status": "ok", "timestamp": 1687981824453, "user": { "displayName": "Myles Harrison", "userId": "02225822412878142066" }, "user_tz": 240 }, "id": "HcjqddW7fdnD", "outputId": "6e611d20-6e7f-4675-9afe-bbe59ac588b2" }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlQAAAHHCAYAAAB5gsZZAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABVHklEQVR4nO3dd1gU5/428HtBWOlIkSYKNkQF9aDYYiBigjVYookhwULUo2LElsQkimgSTNRjOyZGYywnlthLjLGjBhVUrJEoEhELio2uoLvP+4cv83OlM8iy6/25rr2unZlnZr7PDOveTluFEEKAiIiIiCrMQNsFEBEREek6BioiIiIimRioiIiIiGRioCIiIiKSiYGKiIiISCYGKiIiIiKZGKiIiIiIZGKgIiIiIpKJgYqIiIhIJgYqInqlJScnQ6FQYMWKFdoupdrw9/eHv7+/tsuosGnTpkGhUODevXvaLoVeIQxURGW0YsUKKBQKKBQK/Pnnn4WmCyHg6uoKhUKBnj17vpQabt26hWnTpuHMmTNlav98zS++Pvvss5dSY3W1Zs0azJs3r8rW99dff+GDDz6Ai4sLlEolnJ2dERwcjL/++qvKatAlBSGotJcuBz3SbzW0XQCRrqlZsybWrFmD1157TWP8oUOHcOPGDSiVype27lu3biEyMhJubm5o2bJlmeebPn063N3dNcY1b968kqur3tasWYMLFy4gPDz8pa9r8+bNGDhwIGxsbBAaGgp3d3ckJydj2bJl2LhxI9atW4c+ffq89Doqas+ePVW+zr59+6Jhw4bScHZ2NkaOHIk+ffqgb9++0ngHB4cqr42oLBioiMqpe/fu2LBhAxYsWIAaNf7vI7RmzRr4+PhUy9MM3bp1Q+vWrcvU9vHjxzA2NoaBAQ9gV0RSUhI+/PBD1K9fH4cPH4a9vb00bezYsejUqRM+/PBDnDt3DvXr19dipYXl5ubC1NQUxsbGVb5ub29veHt7S8P37t3DyJEj4e3tjQ8++KDK6yEqL/6LSVROAwcOxP3797F3715pXH5+PjZu3Ij333+/yHlycnIwYcIEuLq6QqlUwsPDA7Nnz4YQQqPd3r178dprr8Ha2hrm5ubw8PDA559/DgCIjo5GmzZtAABDhgyRToHIufYnOjoaCoUC69atw5dffgkXFxeYmpoiMzMTABAbG4uuXbvCysoKpqam8PPzQ0xMTKHl/Pnnn2jTpg1q1qyJBg0a4Mcff5RO4RQo6VolhUKBadOmaYy7efMmhg4dCgcHByiVSjRr1gw///xzkfWvX78eX3/9NerUqYOaNWsiICAAV65ckdr5+/tj586duHbtmrTd3Nzcitwmy5cvh0KhwOnTpwtN++abb2BoaIibN28Wt0kxa9Ys5ObmYsmSJRphCgDs7Ozw448/IicnB999912h/oaGhsLZ2RlKpRLu7u4YOXIk8vPzpTbp6ekYN24c3NzcoFQqUadOHYSEhEghvuAUb3JycpHbKTo6WmObNG/eHKdOncLrr78OU1NT6W/txWuoyrqdCyxatAj169eHiYkJfH19ceTIkUq7LuvAgQPo1KkTzMzMYG1tjaCgICQkJJQ637Vr19CwYUM0b94cd+7cAfBse4aHh0ufy4YNG+Lbb7+FWq2W5iv4u509ezaWLFmCBg0aQKlUok2bNjhx4oTGOm7fvo0hQ4agTp06UCqVcHJyQlBQUKH9QfqJR6iIysnNzQ3t27fH2rVr0a1bNwDArl27kJGRgffeew8LFizQaC+EwNtvv42DBw8iNDQULVu2xO7duzFp0iTcvHkTc+fOBfDsmpuePXvC29sb06dPh1KpxJUrV6QA4+npienTp2Pq1KkYPnw4OnXqBADo0KFDqTVnZGQUOnJmZ2cnvZ8xYwaMjY0xceJE5OXlwdjYGAcOHEC3bt3g4+ODiIgIGBgYYPny5ejcuTOOHDkCX19fAMD58+fx1ltvwd7eHtOmTcPTp08REREh69TMnTt30K5dOygUCoSFhcHe3h67du1CaGgoMjMzC522mzlzJgwMDDBx4kRkZGTgu+++Q3BwMGJjYwEAX3zxBTIyMnDjxg1pe5ubmxe57nfeeQejR4/G6tWr0apVK41pq1evhr+/P1xcXIqtfceOHXBzc5P2z4tef/11uLm5YefOndK4W7duwdfXF+np6Rg+fDiaNGmCmzdvYuPGjcjNzYWxsTGys7PRqVMnJCQkYOjQofjXv/6Fe/fuYfv27bhx44bG/iyr+/fvo1u3bnjvvffwwQcflLrPStvOAPDDDz8gLCwMnTp1wrhx45CcnIzevXujVq1aqFOnTrlrfN6+ffvQrVs31K9fH9OmTcOjR4+wcOFCdOzYEfHx8cWG5KSkJHTu3Bk2NjbYu3cv7OzskJubCz8/P9y8eRMjRoxA3bp1cfToUUyePBmpqamFrrdbs2YNsrKyMGLECCgUCnz33Xfo27cv/vnnHxgZGQEA+vXrh7/++gtjxoyBm5sb0tLSsHfvXqSkpBRbG+kRQURlsnz5cgFAnDhxQvz3v/8VFhYWIjc3VwghRP/+/cUbb7whhBCiXr16okePHtJ8W7duFQDEV199pbG8d955RygUCnHlyhUhhBBz584VAMTdu3eLreHEiRMCgFi+fHm5ai7qJYQQBw8eFABE/fr1pb4IIYRarRaNGjUSgYGBQq1WS+Nzc3OFu7u7ePPNN6VxvXv3FjVr1hTXrl2Txl28eFEYGhqK5/+JuXr1arG1AxARERHScGhoqHBychL37t3TaPfee+8JKysrqdaC+j09PUVeXp7Ubv78+QKAOH/+vDSuR48eol69eoXWXVRdAwcOFM7OzkKlUknj4uPjS9326enpAoAICgoqto0QQrz99tsCgMjMzBRCCBESEiIMDAzEiRMnCrUt2P5Tp04VAMTmzZuLbVOwv69evaoxvWA7HTx4UBrn5+cnAIjFixcXWp6fn5/w8/MrNH9p2zkvL0/Y2tqKNm3aiCdPnkjtVqxYIQBoLLM0d+/eLfR30bJlS1G7dm1x//59adzZs2eFgYGBCAkJkcZFRERIn6WEhATh7Ows2rRpIx48eCC1mTFjhjAzMxOXL1/WWO9nn30mDA0NRUpKihDi//4+bG1tNebftm2bACB27NghhBDi4cOHAoCYNWtWmftI+oWn/IgqYMCAAXj06BF+++03ZGVl4bfffiv2dN/vv/8OQ0NDfPzxxxrjJ0yYACEEdu3aBQCwtrYGAGzbtk3jlENlWLRoEfbu3avxet6gQYNgYmIiDZ85cwaJiYl4//33cf/+fdy7dw/37t1DTk4OAgICcPjwYajVaqhUKuzevRu9e/dG3bp1pfk9PT0RGBhYoVqFENi0aRN69eoFIYS07nv37iEwMBAZGRmIj4/XmGfIkCEa1/0UHB36559/KlRDSEgIbt26hYMHD0rjVq9eDRMTE/Tr16/Y+bKysgAAFhYWJS6/YHpmZibUajW2bt2KXr16FXmdW8Fp002bNqFFixZFXsz+/KnV8lAqlRgyZEiZ25e2nU+ePIn79+9j2LBhGtcXBgcHo1atWhWqsUBqairOnDmDwYMHw8bGRhrv7e2NN998E7///nuheS5cuAA/Pz+4ublh3759GjVs2LABnTp1Qq1atTT+xrp06QKVSoXDhw9rLOvdd9/VmP/FvpuYmMDY2BjR0dF4+PChrL6SbuIpP6IKsLe3R5cuXbBmzRrk5uZCpVLhnXfeKbLttWvX4OzsXOhL1tPTU5oOPPsH+6effsJHH32Ezz77DAEBAejbty/eeecd2ReI+/r6lnhR+ot3ACYmJgJ4FrSKk5GRgby8PDx69AiNGjUqNN3Dw6PIL7nS3L17F+np6ViyZAmWLFlSZJu0tDSN4efDHADpi6+iX2xvvvkmnJycsHr1agQEBECtVmPt2rUICgoqMSwVTCsIVsV5PnjdvXsXmZmZpd51mZSUVGKYqwgXF5dyXYBe2nYu+Ft+/m49AKhRo4bsU14Fy/bw8Cg0zdPTE7t370ZOTg7MzMyk8b169YKDgwN2795d6BRvYmIizp07V+g6twLl/RtTKpX49ttvMWHCBDg4OKBdu3bo2bMnQkJC4OjoWM7eki5ioCKqoPfffx/Dhg3D7du30a1bN+kIU0WZmJjg8OHDOHjwIHbu3Ik//vgDv/76Kzp37ow9e/bA0NCwcgovZt3PKzhCNmvWrGIfz2Bubo68vLwyr6O4oygqlarIdX/wwQfFBrrn7wYDUOy2ES9c9F9WhoaGeP/997F06VJ8//33iImJwa1bt0q928zKygpOTk44d+5cie3OnTsHFxcXWFpa4tGjRxWqsShl3cYFXtzvpans7fyy9evXDytXrsTq1asxYsQIjWlqtRpvvvkmPvnkkyLnbdy4scZwWfoeHh6OXr16YevWrdi9ezemTJmCqKgoHDhwoND1eKR/GKiIKqhPnz4YMWIEjh8/jl9//bXYdvXq1cO+ffuQlZWlcXTj77//lqYXMDAwQEBAAAICAvCf//wH33zzDb744gscPHgQXbp0qfCpnfJq0KABAMDS0hJdunQptp29vT1MTEykI1rPu3TpksZwwf/o09PTNcYXHHl4fpkWFhZQqVQlrru8yrvtQkJCMGfOHOzYsQO7du2Cvb19mU5j9uzZE0uXLsWff/5Z6FllAHDkyBEkJydLX/D29vawtLTEhQsXSlxugwYNSm1T1m38shT8LV+5cgVvvPGGNP7p06dITk4uFIQrsuwX/66AZ58lOzs7jaNTwLP/ENSoUQOjRo2ChYWFxmn5Bg0aIDs7u1L/xgqWO2HCBEyYMAGJiYlo2bIl5syZg19++aVS10PVD6+hIqogc3Nz/PDDD5g2bRp69epVbLvu3btDpVLhv//9r8b4uXPnQqFQSHcKPnjwoNC8BUeHCo4EFXxhvPiFWdl8fHzQoEEDzJ49G9nZ2YWm3717F8Cz/7UHBgZi69atSElJkaYnJCRg9+7dGvNYWlrCzs6u0LUp33//vcawoaEh+vXrh02bNhUZIArWXV5mZmbIyMgoc/uC5yL99NNP2LRpE9577z2N64KKM2nSJJiYmGDEiBG4f/++xrQHDx7g3//+N0xNTTFp0iQAz0J07969sWPHDpw8ebLQ8gqOgPTr1w9nz57Fli1bim1TEISf38YqlarYU6eVrXXr1rC1tcXSpUvx9OlTafzq1atlX1fk5OSEli1bYuXKlRp//xcuXMCePXvQvXv3QvMoFAosWbIE77zzDgYNGoTt27dL0wYMGIBjx44V+jsFnn2+nq+/LHJzc/H48WONcQ0aNICFhUW5juSS7uIRKiIZSrrGqECvXr3wxhtv4IsvvkBycjJatGiBPXv2YNu2bQgPD5e+BKdPn47Dhw+jR48eqFevHtLS0vD999+jTp060pGOBg0awNraGosXL4aFhQXMzMzQtm3bQtdAyWVgYICffvoJ3bp1Q7NmzTBkyBC4uLjg5s2bOHjwICwtLbFjxw4AQGRkJP744w906tQJo0aNwtOnT7Fw4UI0a9as0Kmvjz76CDNnzsRHH32E1q1b4/Dhw7h8+XKh9c+cORMHDx5E27ZtMWzYMDRt2hQPHjxAfHw89u3bV2T4LI2Pjw9+/fVXjB8/Hm3atIG5uXmJQRh4dpRq4sSJAFDmh0s2atQIK1euRHBwMLy8vAo9Kf3evXtYu3attN+BZ8+32rNnD/z8/DB8+HB4enoiNTUVGzZswJ9//glra2tMmjQJGzduRP/+/TF06FD4+PjgwYMH2L59OxYvXowWLVqgWbNmaNeuHSZPnowHDx7AxsYG69atK3c4qChjY2NMmzYNY8aMQefOnTFgwAAkJydjxYoVaNCggewjrLNmzUK3bt3Qvn17hIaGSo9NsLKyKvQcswIGBgb45Zdf0Lt3bwwYMAC///47OnfujEmTJmH79u3o2bMnBg8eDB8fH+Tk5OD8+fPYuHEjkpOTy/UoisuXLyMgIAADBgxA06ZNUaNGDWzZsgV37tzBe++9J6vfpCO0d4MhkW55/rEJJXnxsQlCCJGVlSXGjRsnnJ2dhZGRkWjUqJGYNWuWxiMJ9u/fL4KCgoSzs7MwNjYWzs7OYuDAgYVu6962bZto2rSpqFGjRqm38ZdWc8Ht8Bs2bChy+unTp0Xfvn2Fra2tUCqVol69emLAgAFi//79Gu0OHTokfHx8hLGxsahfv75YvHixdOv683Jzc0VoaKiwsrISFhYWYsCAASItLa3Q7fFCCHHnzh0xevRo4erqKoyMjISjo6MICAgQS5YsKbX+oh6FkJ2dLd5//31hbW0tAEiPUCjpcQ6pqanC0NBQNG7cuMjtU5Jz586JgQMHCicnJ6n+gQMHajzK4XnXrl0TISEhwt7eXiiVSlG/fn0xevRojccU3L9/X4SFhQkXFxdhbGws6tSpIwYNGqTxeImkpCTRpUsXoVQqhYODg/j888/F3r17i3xsQrNmzYqspbjHJpRlOwshxIIFC0S9evWEUqkUvr6+IiYmRvj4+IiuXbuWcesV/dgEIYTYt2+f6NixozAxMRGWlpaiV69e4uLFixptnn9sQoHc3Fzh5+cnzM3NxfHjx4UQzz6XkydPFg0bNhTGxsbCzs5OdOjQQcyePVvk5+dr9LGoxyE8X9+9e/fE6NGjRZMmTYSZmZmwsrISbdu2FevXry9zn0m3KYSoplcTEpFOmzZtGiIjI6vtBctlce/ePTg5OWHq1KmYMmWKtsvRWWq1Gvb29ujbty+WLl2q7XKIXgpeQ0VEVIwVK1ZApVLhww8/1HYpOuPx48eFQvSqVavw4MGDSvnpGaLqitdQERG94MCBA7h48SK+/vpr9O7dmz8bUg7Hjx/HuHHj0L9/f9ja2iI+Ph7Lli1D8+bN0b9/f22XR/TSMFAREb1g+vTpOHr0KDp27IiFCxdquxyd4ubmBldXVyxYsEC6MD4kJAQzZ84s10NEiXQNr6EiIiIikonXUBERERHJxEBFREREJBOvoaoCarUat27dgoWFRZX9dAgRERHJI4RAVlYWnJ2dS/2RegaqKnDr1i24urpquwwiIiKqgOvXr6NOnToltmGgqgIFP4h7/fp1WFpaarkaIiIiKovMzEy4urpq/LB9cRioqkDBaT5LS0sGKiIiIh1Tlst1eFE6ERERkUwMVEREREQyMVARERERycRARURERCQTAxURERGRTAxURERERDLxsQlVqHnEbhgoTbVdBhERkV5JntlD2yXwCBURERGRXAxURERERDIxUBERERHJpNOByt/fH+Hh4doug4iIiF5xOh2oiIiIiKoDBqoS5Ofna7sEIiIi0gF6E6jy8vIwceJEuLi4wMzMDG3btkV0dLQ0/f79+xg4cCBcXFxgamoKLy8vrF27VmMZ/v7+CAsLQ3h4OOzs7BAYGIjo6GgoFArs378frVu3hqmpKTp06IBLly5VcQ+JiIioutKbQBUWFoZjx45h3bp1OHfuHPr374+uXbsiMTERAPD48WP4+Phg586duHDhAoYPH44PP/wQcXFxGstZuXIljI2NERMTg8WLF0vjv/jiC8yZMwcnT55EjRo1MHTo0CrtHxEREVVfCiGE0HYRFeXv74+WLVti/PjxqF+/PlJSUuDs7CxN79KlC3x9ffHNN98UOX/Pnj3RpEkTzJ49W1peZmYm4uPjpTbR0dF44403sG/fPgQEBAAAfv/9d/To0QOPHj1CzZo1Cy03Ly8PeXl50nBmZiZcXV3hGr6eD/YkIiKqZC/rwZ6ZmZmwsrJCRkYGLC0tS2yrF09KP3/+PFQqFRo3bqwxPi8vD7a2tgAAlUqFb775BuvXr8fNmzeRn5+PvLw8mJpqBhwfH58i1+Ht7S29d3JyAgCkpaWhbt26hdpGRUUhMjJSVp+IiIhId+hFoMrOzoahoSFOnToFQ0NDjWnm5uYAgFmzZmH+/PmYN28evLy8YGZmhvDw8EIXnpuZmRW5DiMjI+m9QqEAAKjV6iLbTp48GePHj5eGC45QERERkX7Si0DVqlUrqFQqpKWloVOnTkW2iYmJQVBQED744AMAz8LQ5cuX0bRp00qvR6lUQqlUVvpyiYiIqHrSi4vSGzdujODgYISEhGDz5s24evUq4uLiEBUVhZ07dwIAGjVqhL179+Lo0aNISEjAiBEjcOfOHS1XTkRERPpALwIVACxfvhwhISGYMGECPDw80Lt3b5w4cUK6xunLL7/Ev/71LwQGBsLf3x+Ojo7o3bu3dosmIiIivaDTd/npioK7BHiXHxERUeWrDnf56c0RKiIiIiJtYaAiIiIikomBioiIiEgmvXhsgq64EBlY6jlYIiIi0j08QkVEREQkEwMVERERkUwMVEREREQyMVARERERycRARURERCQTAxURERGRTAxURERERDIxUBERERHJxEBFREREJBMDFREREZFMDFREREREMjFQEREREcnEQEVEREQkEwMVERERkUwMVEREREQyMVARERERycRARURERCRTDW0X8CppHrEbBkpTbZdBRERUpZJn9tB2CS8dj1ARERERycRARURERCQTAxURERGRTAxUAPz9/REeHl7m9itWrIC1tfVLq4eIiIh0CwMVERERkUwMVEREREQyVetA5e/vjzFjxiA8PBy1atWCg4MDli5dipycHAwZMgQWFhZo2LAhdu3aJc1z6NAh+Pr6QqlUwsnJCZ999hmePn0qTc/JyUFISAjMzc3h5OSEOXPmFFpvXl4eJk6cCBcXF5iZmaFt27aIjo6uii4TERGRDqrWgQoAVq5cCTs7O8TFxWHMmDEYOXIk+vfvjw4dOiA+Ph5vvfUWPvzwQ+Tm5uLmzZvo3r072rRpg7Nnz+KHH37AsmXL8NVXX0nLmzRpEg4dOoRt27Zhz549iI6ORnx8vMY6w8LCcOzYMaxbtw7nzp1D//790bVrVyQmJlZ194mIiEgHKIQQQttFFMff3x8qlQpHjhwBAKhUKlhZWaFv375YtWoVAOD27dtwcnLCsWPHsGPHDmzatAkJCQlQKBQAgO+//x6ffvopMjIykJubC1tbW/zyyy/o378/AODBgweoU6cOhg8fjnnz5iElJQX169dHSkoKnJ2dpVq6dOkCX19ffPPNN1ixYgXCw8ORnp5eZN15eXnIy8uThjMzM+Hq6grX8PV8sCcREb1ydPXBnpmZmbCyskJGRgYsLS1LbFvtn5Tu7e0tvTc0NIStrS28vLykcQ4ODgCAtLQ0JCQkoH379lKYAoCOHTsiOzsbN27cwMOHD5Gfn4+2bdtK021sbODh4SENnz9/HiqVCo0bN9aoIy8vD7a2tmWqOSoqCpGRkeXrKBEREemsah+ojIyMNIYVCoXGuILwpFarK2V92dnZMDQ0xKlTp2BoaKgxzdzcvEzLmDx5MsaPHy8NFxyhIiIiIv1U7QNVeXh6emLTpk0QQkhBKyYmBhYWFqhTpw5sbGxgZGSE2NhY1K1bFwDw8OFDXL58GX5+fgCAVq1aQaVSIS0tDZ06dapQHUqlEkqlsnI6RURERNVetb8ovTxGjRqF69evY8yYMfj777+xbds2REREYPz48TAwMIC5uTlCQ0MxadIkHDhwABcuXMDgwYNhYPB/m6Fx48YIDg5GSEgINm/ejKtXryIuLg5RUVHYuXOnFntHRERE1ZVeHaFycXHB77//jkmTJqFFixawsbFBaGgovvzyS6nNrFmzkJ2djV69esHCwgITJkxARkaGxnKWL1+Or776ChMmTMDNmzdhZ2eHdu3aoWfPnlXdJSIiItIB1fouP31RcJcA7/IjIqJX0atwl59enfIjIiIi0gYGKiIiIiKZGKiIiIiIZNKri9KruwuRgaWegyUiIiLdwyNURERERDIxUBERERHJxEBFREREJBMDFREREZFMDFREREREMjFQEREREcnEQEVEREQkEwMVERERkUwMVEREREQyMVARERERycRARURERCQTAxURERGRTAxURERERDIxUBERERHJxEBFREREJBMDFREREZFMDFREREREMtXQdgGvkuYRu2GgNNV2GUREr6zkmT20XQLpKR6hIiIiIpKJgYqIiIhIJgYqIiIiIpleuUClUCiwdevWYqdHR0dDoVAgPT29ymoiIiIi3fbKBarSdOjQAampqbCystJ2KURERKQjeJffC4yNjeHo6KjtMoiIiEiHaPUIlb+/P8aMGYPw8HDUqlULDg4OWLp0KXJycjBkyBBYWFigYcOG2LVrFwBApVIhNDQU7u7uMDExgYeHB+bPn19ouT///DOaNWsGpVIJJycnhIWFaUy/d+8e+vTpA1NTUzRq1Ajbt2+Xpr14ym/FihWwtrbG7t274enpCXNzc3Tt2hWpqakvb8MQERGRTtH6Kb+VK1fCzs4OcXFxGDNmDEaOHIn+/fujQ4cOiI+Px1tvvYUPP/wQubm5UKvVqFOnDjZs2ICLFy9i6tSp+Pzzz7F+/XppeT/88ANGjx6N4cOH4/z589i+fTsaNmyosc7IyEgMGDAA586dQ/fu3REcHIwHDx4UW2Nubi5mz56N//3vfzh8+DBSUlIwceLEl7ZNiIiISLcohBBCWyv39/eHSqXCkSNHADw7AmVlZYW+ffti1apVAIDbt2/DyckJx44dQ7t27QotIywsDLdv38bGjRsBAC4uLhgyZAi++uqrItepUCjw5ZdfYsaMGQCAnJwcmJubY9euXejatSuio6Pxxhtv4OHDh7C2tsaKFSswZMgQXLlyBQ0aNAAAfP/995g+fTpu375d5Dry8vKQl5cnDWdmZsLV1RWu4ev5YE8iIi3igz2pPDIzM2FlZYWMjAxYWlqW2FbrR6i8vb2l94aGhrC1tYWXl5c0zsHBAQCQlpYGAFi0aBF8fHxgb28Pc3NzLFmyBCkpKVKbW7duISAgoMzrNDMzg6WlpbT8opiamkphCgCcnJxKbB8VFQUrKyvp5erqWmI9REREpNu0HqiMjIw0hhUKhcY4hUIBAFCr1Vi3bh0mTpyI0NBQ7NmzB2fOnMGQIUOQn58PADAxManwOtVqdbnal3Rgb/LkycjIyJBe169fL1NdREREpJt06i6/mJgYdOjQAaNGjZLGJSUlSe8tLCzg5uaG/fv344033tBGiQAApVIJpVKptfUTERFR1dKpQNWoUSOsWrUKu3fvhru7O/73v//hxIkTcHd3l9pMmzYN//73v1G7dm1069YNWVlZiImJwZgxY7RYOREREekzrZ/yK48RI0agb9++ePfdd9G2bVvcv39f42gVAAwaNAjz5s3D999/j2bNmqFnz55ITEzUUsVERET0KtDqXX6vioK7BHiXHxGRdvEuPyoPnbrLj4iIiEjXMVARERERycRARURERCSTTt3lp+suRAaWeg6WiIiIdA+PUBERERHJxEBFREREJBMDFREREZFMDFREREREMjFQEREREcnEQEVEREQkEwMVERERkUwMVEREREQyMVARERERycRARURERCQTAxURERGRTAxURERERDIxUBERERHJxEBFREREJBMDFREREZFMDFREREREMjFQEREREclUQ9sFvEqaR+yGgdJU22UQEVVY8swe2i6BqFriESoiIiIimRioiIiIiGRioCIiIiKS6ZULVG5ubpg3b540rFAosHXrVq3VQ0RERLrvlQtUL0pNTUW3bt3K1HbatGlo2bLlyy2IiIiIdI5O3uWXn58PY2PjSlmWo6NjpSyHiIiIXl3V4giVv78/wsLCEBYWBisrK9jZ2WHKlCkQQgB4dppuxowZCAkJgaWlJYYPHw4A+PPPP9GpUyeYmJjA1dUVH3/8MXJycqTlpqWloVevXjAxMYG7uztWr15daN0vnvK7ceMGBg4cCBsbG5iZmaF169aIjY3FihUrEBkZibNnz0KhUEChUGDFihUvdbsQERGRbqg2R6hWrlyJ0NBQxMXF4eTJkxg+fDjq1q2LYcOGAQBmz56NqVOnIiIiAgCQlJSErl274quvvsLPP/+Mu3fvSqFs+fLlAIDBgwfj1q1bOHjwIIyMjPDxxx8jLS2t2Bqys7Ph5+cHFxcXbN++HY6OjoiPj4darca7776LCxcu4I8//sC+ffsAAFZWVkUuJy8vD3l5edJwZmZmpWwjIiIiqp6qTaBydXXF3LlzoVAo4OHhgfPnz2Pu3LlSoOrcuTMmTJggtf/oo48QHByM8PBwAECjRo2wYMEC+Pn54YcffkBKSgp27dqFuLg4tGnTBgCwbNkyeHp6FlvDmjVrcPfuXZw4cQI2NjYAgIYNG0rTzc3NUaNGjVJPE0ZFRSEyMrJC24GIiIh0T7U45QcA7dq1g0KhkIbbt2+PxMREqFQqAEDr1q012p89exYrVqyAubm59AoMDIRarcbVq1eRkJCAGjVqwMfHR5qnSZMmsLa2LraGM2fOoFWrVlKYqqjJkycjIyNDel2/fl3W8oiIiKh6qzZHqEpjZmamMZydnY0RI0bg448/LtS2bt26uHz5crnXYWJiUuH6nqdUKqFUKitlWURERFT9VZtAFRsbqzF8/PhxNGrUCIaGhkW2/9e//oWLFy9qnJJ7XpMmTfD06VOcOnVKOuV36dIlpKenF1uDt7c3fvrpJzx48KDIo1TGxsbSETMiIiKiAtXmlF9KSgrGjx+PS5cuYe3atVi4cCHGjh1bbPtPP/0UR48eRVhYGM6cOYPExERs27YNYWFhAAAPDw907doVI0aMQGxsLE6dOoWPPvqoxKNQAwcOhKOjI3r37o2YmBj8888/2LRpE44dOwbg2d2GV69exZkzZ3Dv3j2NC8+JiIjo1VVtAlVISAgePXoEX19fjB49GmPHjpUej1AUb29vHDp0CJcvX0anTp3QqlUrTJ06Fc7OzlKb5cuXw9nZGX5+fujbty+GDx+O2rVrF7tMY2Nj7NmzB7Vr10b37t3h5eWFmTNnSkfJ+vXrh65du+KNN96Avb091q5dW3kbgIiIiHSWQhQ87EmL/P390bJlS42fhNEnmZmZsLKygmv4ehgoTbVdDhFRhSXP7KHtEoiqTMH3d0ZGBiwtLUtsW22OUBERERHpKgYqIiIiIpmqxV1+0dHR2i6BiIiIqMKqRaB6VVyIDCz1HCwRERHpHp7yIyIiIpKJgYqIiIhIJgYqIiIiIpkYqIiIiIhkYqAiIiIikomBioiIiEgmBioiIiIimRioiIiIiGRioCIiIiKSiYGKiIiISCYGKiIiIiKZGKiIiIiIZGKgIiIiIpKJgYqIiIhIJgYqIiIiIpkYqIiIiIhkYqAiIiIikqmGtgt4lTSP2A0Dpam2yyAiLUue2UPbJRBRJeMRKiIiIiKZGKiIiIiIZGKgIiIiIpJJ7wOVv78/wsPDtV0GERER6TG9D1TlER0dDYVCgfT0dG2XQkRERDqEgYqIiIhIJr0KVDk5OQgJCYG5uTmcnJwwZ84cjen/+9//0Lp1a1hYWMDR0RHvv/8+0tLSAADJycl44403AAC1atWCQqHA4MGDAQB//PEHXnvtNVhbW8PW1hY9e/ZEUlJSlfaNiIiIqi+9ClSTJk3CoUOHsG3bNuzZswfR0dGIj4+Xpj958gQzZszA2bNnsXXrViQnJ0uhydXVFZs2bQIAXLp0CampqZg/fz6AZ0Ft/PjxOHnyJPbv3w8DAwP06dMHarW6yDry8vKQmZmp8SIiIiL9pTcP9szOzsayZcvwyy+/ICAgAACwcuVK1KlTR2ozdOhQ6X39+vWxYMECtGnTBtnZ2TA3N4eNjQ0AoHbt2rC2tpba9uvXT2NdP//8M+zt7XHx4kU0b968UC1RUVGIjIyszO4RERFRNVahQJWeno64uDikpaUVOkoTEhJSKYWVV1JSEvLz89G2bVtpnI2NDTw8PKThU6dOYdq0aTh79iwePnwo1Z6SkoKmTZsWu+zExERMnToVsbGxuHfvnsZ8RQWqyZMnY/z48dJwZmYmXF1dZfeRiIiIqqdyB6odO3YgODgY2dnZsLS0hEKhkKYpFAqtBarS5OTkIDAwEIGBgVi9ejXs7e2RkpKCwMBA5Ofnlzhvr169UK9ePSxduhTOzs5Qq9Vo3rx5sfMplUoolcqX0Q0iIiKqhsp9DdWECRMwdOhQZGdnIz09HQ8fPpReDx48eBk1lkmDBg1gZGSE2NhYadzDhw9x+fJlAMDff/+N+/fvY+bMmejUqROaNGkiXZBewNjYGACgUqmkcffv38elS5fw5ZdfIiAgAJ6ennj48GEV9IiIiIh0RbkD1c2bN/Hxxx/D1LR6/civubk5QkNDMWnSJBw4cAAXLlzA4MGDYWDwrIt169aFsbExFi5ciH/++Qfbt2/HjBkzNJZRr149KBQK/Pbbb7h79y6ys7NRq1Yt2NraYsmSJbhy5QoOHDigcTqPiIiIqNyBKjAwECdPnnwZtcg2a9YsdOrUCb169UKXLl3w2muvwcfHBwBgb2+PFStWYMOGDWjatClmzpyJ2bNna8zv4uKCyMhIfPbZZ3BwcEBYWBgMDAywbt06nDp1Cs2bN8e4ceMwa9YsbXSPiIiIqimFEEKUZ4Zly5Zh+vTpGDJkCLy8vGBkZKQx/e23367UAvVBZmYmrKys4Bq+HgbK6nVkj4iqXvLMHtougYjKoOD7OyMjA5aWliW2LfdF6cOGDQMATJ8+vdA0hUKhcf0RERER0aug3IGquIdZEhEREb2qZD0p/fHjx5VVBxEREZHOKvcRKpVKhW+++QaLFy/GnTt3cPnyZdSvXx9TpkyBm5sbQkNDX0adeuFCZGCp52CJiIhI95T7CNXXX3+NFStW4LvvvpOe2wQAzZs3x08//VSpxRERERHpgnIHqlWrVmHJkiUIDg6GoaGhNL5Fixb4+++/K7U4IiIiIl1QoQd7NmzYsNB4tVqNJ0+eVEpRRERERLqk3IGqadOmOHLkSKHxGzduRKtWrSqlKCIiIiJdUu6L0qdOnYpBgwbh5s2bUKvV2Lx5My5duoRVq1bht99+exk1EhEREVVr5T5CFRQUhB07dmDfvn0wMzPD1KlTkZCQgB07duDNN998GTUSERERVWvlPkJ148YNdOrUCXv37i007fjx42jXrl2lFEZERESkK8p9hOqtt97CgwcPCo2PiYlB165dK6UoIiIiIl1S7kDVrl07vPXWW8jKypLGHT58GN27d0dERESlFkdERESkC8odqH766SfUrVsXvXr1Ql5eHg4ePIgePXpg+vTpGDdu3MuokYiIiKhaK3egMjAwwLp162BkZITOnTvj7bffRlRUFMaOHfsy6iMiIiKq9hRCCFFao3PnzhUal5WVhYEDB6JHjx4YOXKkNN7b27tyK9QDmZmZsLKyQkZGBn/Lj4iISEeU5/u7TIHKwMAACoUCzzd9frjgvUKhgEqlklm+/mGgIiIi0j3l+f4u02MTrl69WimFEREREemjMgWqevXqvew6iIiIiHRWuR/sCQBJSUmYN28eEhISADz7fb+xY8eiQYMGlVocERERkS4o911+u3fvRtOmTREXFwdvb294e3sjNjYWzZo1K/Lp6URERET6rkwXpT+vVatWCAwMxMyZMzXGf/bZZ9izZw/i4+MrtUB9UHBRm2v4ehgoTbVdDpFeSJ7ZQ9slEJGeK89F6eU+QpWQkIDQ0NBC44cOHYqLFy+Wd3FEREREOq/cgcre3h5nzpwpNP7MmTOoXbt2ZdREREREpFPKfFH69OnTMXHiRAwbNgzDhw/HP//8gw4dOgB49sPI3377LcaPH//SCiUiIiKqrsp8DZWhoSFSU1Nhb2+PefPmYc6cObh16xYAwNnZGZMmTcLHH38MhULxUgsuir+/P1q2bIl58+ZV+brLgtdQEVU+XkNFRC9bpT/YE4DGU9HHjRuHcePGISsrCwBgYWEho1wiIiIi3Vau51C9ePSJQYqIiIionBelN27cGDY2NiW+tEWtVuOTTz6BjY0NHB0dMW3aNGlaSkoKgoKCYG5uDktLSwwYMAB37tyRpg8ePBi9e/fWWF54eDj8/f2l4Y0bN8LLywsmJiawtbVFly5dkJOT85J7RURERLqgXEeoIiMjYWVl9bJqkWXlypUYP348YmNjcezYMQwePBgdO3ZEQECAFKYOHTqEp0+fYvTo0Xj33XcRHR1dpmWnpqZi4MCB+O6779CnTx9kZWXhyJEjKO7ys7y8POTl5UnDmZmZldFFIiIiqqbKFajee++9avtoBG9vb0RERAAAGjVqhP/+97/Yv38/AOD8+fO4evUqXF1dAQCrVq1Cs2bNcOLECbRp06bUZaempuLp06fo27ev9LuGXl5exbaPiopCZGSk3C4RERGRjijzKT9t3L1XHt7e3hrDTk5OSEtLQ0JCAlxdXaUwBTz77UFra2vptwhL06JFCwQEBMDLywv9+/fH0qVL8fDhw2LbT548GRkZGdLr+vXrFesUERER6YQyB6py/kJNlTMyMtIYVigUUKvVZZrXwMCgUP+ePHkivTc0NMTevXuxa9cuNG3aFAsXLoSHhweuXr1a5PKUSiUsLS01XkRERKS/yhyo1Gp1tT3dVxJPT09cv35d4yjRxYsXkZ6ejqZNmwJ49vT31NRUjflefBq8QqFAx44dERkZidOnT8PY2Bhbtmx56fUTERFR9Vfun57RNV26dIGXlxeCg4MRHx+PuLg4hISEwM/PD61btwYAdO7cGSdPnsSqVauQmJiIiIgIXLhwQVpGbGwsvvnmG5w8eRIpKSnYvHkz7t69C09PT211i4iIiKoRvQ9UCoUC27ZtQ61atfD666+jS5cuqF+/Pn799VepTWBgIKZMmYJPPvkEbdq0QVZWFkJCQqTplpaWOHz4MLp3747GjRvjyy+/xJw5c9CtWzdtdImIiIiqmTL/9AxVHH96hqjy8adniOhlK89Pz+j9ESoiIiKil42BioiIiEgmBioiIiIimcr1pHSS50JkIJ9JRUREpId4hIqIiIhIJgYqIiIiIpkYqIiIiIhkYqAiIiIikomBioiIiEgmBioiIiIimRioiIiIiGRioCIiIiKSiYGKiIiISCYGKiIiIiKZGKiIiIiIZGKgIiIiIpKJgYqIiIhIJgYqIiIiIpkYqIiIiIhkYqAiIiIikomBioiIiEimGtou4FXSPGI3DJSm2i6DqEokz+yh7RKIiKoMj1ARERERycRARURERCQTAxURERGRTAxUZTBt2jS0bNlS22UQERFRNaWXgcrf3x/h4eHaLoOIiIheEXoZqIiIiIiqkt4FqsGDB+PQoUOYP38+FAoFFAoFkpKSEBoaCnd3d5iYmMDDwwPz58/XmC86Ohq+vr4wMzODtbU1OnbsiGvXrhW5jqSkJNSvXx9hYWEQQlRFt4iIiKga07vnUM2fPx+XL19G8+bNMX36dABArVq1UKdOHWzYsAG2trY4evQohg8fDicnJwwYMABPnz5F7969MWzYMKxduxb5+fmIi4uDQqEotPxz584hMDAQoaGh+Oqrr4qsIS8vD3l5edJwZmbmy+ksERERVQt6F6isrKxgbGwMU1NTODo6SuMjIyOl9+7u7jh27BjWr1+PAQMGIDMzExkZGejZsycaNGgAAPD09Cy07KNHj6Jnz5744osvMGHChGJriIqK0lgfERER6Te9O+VXnEWLFsHHxwf29vYwNzfHkiVLkJKSAgCwsbHB4MGDERgYiF69emH+/PlITU3VmD8lJQVvvvkmpk6dWmKYAoDJkycjIyNDel2/fv2l9YuIiIi075UIVOvWrcPEiRMRGhqKPXv24MyZMxgyZAjy8/OlNsuXL8exY8fQoUMH/Prrr2jcuDGOHz8uTbe3t4evry/Wrl1b6ik8pVIJS0tLjRcRERHpL70MVMbGxlCpVNJwTEwMOnTogFGjRqFVq1Zo2LAhkpKSCs3XqlUrTJ48GUePHkXz5s2xZs0aaZqJiQl+++031KxZE4GBgcjKyqqSvhAREVH1p5eBys3NDbGxsUhOTsa9e/fQqFEjnDx5Ert378bly5cxZcoUnDhxQmp/9epVTJ48GceOHcO1a9ewZ88eJCYmFrqOyszMDDt37kSNGjXQrVs3ZGdnV3XXiIiIqBrSy0A1ceJEGBoaomnTprC3t0dgYCD69u2Ld999F23btsX9+/cxatQoqb2pqSn+/vtv9OvXD40bN8bw4cMxevRojBgxotCyzc3NsWvXLggh0KNHD+Tk5FRl14iIiKgaUgg+SOmly8zMhJWVFVzD18NAaartcoiqRPLMHtougYhIloLv74yMjFKvh9bLI1REREREVYmBioiIiEgmBioiIiIimfTuSenV2YXIQD6TioiISA/xCBURERGRTAxURERERDIxUBERERHJxEBFREREJBMDFREREZFMDFREREREMjFQEREREcnEQEVEREQkEwMVERERkUwMVEREREQyMVARERERycRARURERCQTAxURERGRTAxURERERDIxUBERERHJxEBFREREJBMDFREREZFMNbRdwKukecRuGChNtV0GVaLkmT20XQIREVUDPEJFREREJBMDFREREZFMDFREREREMr1ygSo5ORkKhQJnzpzRdilERESkJ165i9JdXV2RmpoKOzs7bZdCREREeuKVClT5+fkwNjaGo6OjtkshIiIiPaLTp/z8/f0RFhaGsLAwWFlZwc7ODlOmTIEQAgDg5uaGGTNmICQkBJaWlhg+fHiRp/z++usv9OzZE5aWlrCwsECnTp2QlJQkTf/pp5/g6emJmjVrokmTJvj++++ruqtERERUjen8EaqVK1ciNDQUcXFxOHnyJIYPH466deti2LBhAIDZs2dj6tSpiIiIKHL+mzdv4vXXX4e/vz8OHDgAS0tLxMTE4OnTpwCA1atXY+rUqfjvf/+LVq1a4fTp0xg2bBjMzMwwaNCgIpeZl5eHvLw8aTgzM7OSe01ERETVic4HKldXV8ydOxcKhQIeHh44f/485s6dKwWqzp07Y8KECVL75ORkjfkXLVoEKysrrFu3DkZGRgCAxo0bS9MjIiIwZ84c9O3bFwDg7u6Oixcv4scffyw2UEVFRSEyMrIyu0lERETVmE6f8gOAdu3aQaFQSMPt27dHYmIiVCoVAKB169Ylzn/mzBl06tRJClPPy8nJQVJSEkJDQ2Fubi69vvrqK41Tgi+aPHkyMjIypNf169cr2DsiIiLSBTp/hKo0ZmZmJU43MTEpdlp2djYAYOnSpWjbtq3GNENDw2LnUyqVUCqV5aiSiIiIdJnOB6rY2FiN4ePHj6NRo0YlBp7neXt7Y+XKlXjy5Emho1QODg5wdnbGP//8g+Dg4EqrmYiIiPSLzp/yS0lJwfjx43Hp0iWsXbsWCxcuxNixY8s8f1hYGDIzM/Hee+/h5MmTSExMxP/+9z9cunQJABAZGYmoqCgsWLAAly9fxvnz57F8+XL85z//eVldIiIiIh2j80eoQkJC8OjRI/j6+sLQ0BBjx47F8OHDyzy/ra0tDhw4gEmTJsHPzw+GhoZo2bIlOnbsCAD46KOPYGpqilmzZmHSpEkwMzODl5cXwsPDX1KPiIiISNcoRMFDm3SQv78/WrZsiXnz5mm7lBJlZmbCysoKruHrYaA01XY5VImSZ/bQdglERPSSFHx/Z2RkwNLSssS2On/Kj4iIiEjbGKiIiIiIZNLpa6iio6O1XQIRERGRbgcqXXMhMrDUc7BERESke3jKj4iIiEgmBioiIiIimRioiIiIiGRioCIiIiKSiYGKiIiISCYGKiIiIiKZGKiIiIiIZGKgIiIiIpKJgYqIiIhIJgYqIiIiIpkYqIiIiIhkYqAiIiIikomBioiIiEgmBioiIiIimRioiIiIiGRioCIiIiKSiYGKiIiISKYa2i7gVdI8YjcMlKbaLoMqUfLMHtougYiIqgEeoSIiIiKSiYGKiIiISCYGKiIiIiKZXvlANXjwYPTu3VvbZRAREZEOe2UuSk9OToa7uztOnz6Nli1bSuPnz58PIYT2CiMiIiKd98oEquJYWVlpuwQiIiLScTp3yk+tViMqKgru7u4wMTFBixYtsHHjRgDAw4cPERwcDHt7e5iYmKBRo0ZYvnw5AMDd3R0A0KpVKygUCvj7+wMofMrP398fY8aMQXh4OGrVqgUHBwcsXboUOTk5GDJkCCwsLNCwYUPs2rWrSvtNRERE1ZfOHaGKiorCL7/8gsWLF6NRo0Y4fPgwPvjgA9jb22PDhg24ePEidu3aBTs7O1y5cgWPHj0CAMTFxcHX1xf79u1Ds2bNYGxsXOw6Vq5ciU8++QRxcXH49ddfMXLkSGzZsgV9+vTB559/jrlz5+LDDz9ESkoKTE0LP1cqLy8PeXl50nBmZmblbwgiIiKqNhRChy4gysvLg42NDfbt24f27dtL4z/66CPk5uYiOzsbdnZ2+PnnnwvNW9w1VIMHD0Z6ejq2bt0K4NkRKpVKhSNHjgAAVCoVrKys0LdvX6xatQoAcPv2bTg5OeHYsWNo165doXVNmzYNkZGRhca7hq/ngz31DB/sSUSkvzIzM2FlZYWMjAxYWlqW2FanTvlduXIFubm5ePPNN2Fubi69Vq1ahaSkJIwcORLr1q1Dy5Yt8cknn+Do0aMVWo+3t7f03tDQELa2tvDy8pLGOTg4AADS0tKKnH/y5MnIyMiQXtevX69QHURERKQbdOqUX3Z2NgBg586dcHFx0ZimVCrh6uqKa9eu4ffff8fevXsREBCA0aNHY/bs2eVaj5GRkcawQqHQGKdQKAA8u56rKEqlEkqlslzrJCIiIt2lU4GqadOmUCqVSElJgZ+fX5Ft7O3tMWjQIAwaNAidOnXCpEmTMHv2bOmaKZVKVZUlExER0StApwKVhYUFJk6ciHHjxkGtVuO1115DRkYGYmJiYGlpiaSkJPj4+KBZs2bIy8vDb7/9Bk9PTwBA7dq1YWJigj/++AN16tRBzZo1+cgEIiIiqhQ6dQ0VAMyYMQNTpkxBVFQUPD090bVrV+zcuRPu7u4wNjbG5MmT4e3tjddffx2GhoZYt24dAKBGjRpYsGABfvzxRzg7OyMoKEjLPSEiIiJ9oVN3+emqgrsEeJef/uFdfkRE+ktv7/IjIiIiqo4YqIiIiIhkYqAiIiIikkmn7vLTdRciA0s9B0tERES6h0eoiIiIiGRioCIiIiKSiYGKiIiISCYGKiIiIiKZGKiIiIiIZGKgIiIiIpKJgYqIiIhIJgYqIiIiIpkYqIiIiIhkYqAiIiIikomBioiIiEgmBioiIiIimRioiIiIiGRioCIiIiKSiYGKiIiISCYGKiIiIiKZGKiIiIiIZKqh7QJeJc0jdsNAaartMqgSJc/soe0SiIioGuARKiIiIiKZGKiIiIiIZGKgIiIiIpLplQtUgwcPRu/evbVdBhEREemRV+6i9Pnz50MIoe0yiIiISI+8coHKyspK2yUQERGRntHbU34bN26El5cXTExMYGtriy5duiAnJ6fQKb+srCwEBwfDzMwMTk5OmDt3Lvz9/REeHi61cXNzwzfffIOhQ4fCwsICdevWxZIlS6q+U0RERFQt6WWgSk1NxcCBAzF06FAkJCQgOjoaffv2LfJU3/jx4xETE4Pt27dj7969OHLkCOLj4wu1mzNnDlq3bo3Tp09j1KhRGDlyJC5dulTk+vPy8pCZmanxIiIiIv2ll6f8UlNT8fTpU/Tt2xf16tUDAHh5eRVql5WVhZUrV2LNmjUICAgAACxfvhzOzs6F2nbv3h2jRo0CAHz66aeYO3cuDh48CA8Pj0Jto6KiEBkZWZldIiIiompML49QtWjRAgEBAfDy8kL//v2xdOlSPHz4sFC7f/75B0+ePIGvr680zsrKqsiQ5O3tLb1XKBRwdHREWlpakeufPHkyMjIypNf169croVdERERUXelloDI0NMTevXuxa9cuNG3aFAsXLoSHhweuXr1a4WUaGRlpDCsUCqjV6iLbKpVKWFpaaryIiIhIf+lloAKeBZ6OHTsiMjISp0+fhrGxMbZs2aLRpn79+jAyMsKJEyekcRkZGbh8+XJVl0tEREQ6TC+voYqNjcX+/fvx1ltvoXbt2oiNjcXdu3fh6emJc+fOSe0sLCwwaNAgTJo0CTY2NqhduzYiIiJgYGAAhUKhxR4QERGRLtHLI1SWlpY4fPgwunfvjsaNG+PLL7/EnDlz0K1bt0Jt//Of/6B9+/bo2bMnunTpgo4dO8LT0xM1a9bUQuVERESkixSCjw3XkJOTAxcXF8yZMwehoaGVsszMzExYWVnBNXw9DJSmlbJMqh6SZ/bQdglERPSSFHx/Z2RklHo9tF6e8iuP06dP4++//4avry8yMjIwffp0AEBQUJCWKyMiIiJd8coHKgCYPXs2Ll26BGNjY/j4+ODIkSOws7PTdllERESkI3jKrwqU55AhERERVQ/l+f7Wy4vSiYiIiKoSAxURERGRTAxURERERDIxUBERERHJxEBFREREJBMDFREREZFMDFREREREMjFQEREREcnEJ6VXgYJnp2ZmZmq5EiIiIiqrgu/tsjwDnYGqCty/fx8A4OrqquVKiIiIqLyysrJgZWVVYhsGqipgY2MDAEhJSSl1h+i6zMxMuLq64vr163r/MzuvUl+BV6u/7Kt+Yl/108vsqxACWVlZcHZ2LrUtA1UVMDB4dqmalZWV3v9hF7C0tGRf9dSr1F/2VT+xr/rpZfW1rAdCeFE6ERERkUwMVEREREQyMVBVAaVSiYiICCiVSm2X8tKxr/rrVeov+6qf2Ff9VF36qhBluReQiIiIiIrFI1REREREMjFQEREREcnEQEVEREQkEwMVERERkUwMVJVk0aJFcHNzQ82aNdG2bVvExcWV2H7Dhg1o0qQJatasCS8vL/z+++9VVKk8UVFRaNOmDSwsLFC7dm307t0bly5dKnGeFStWQKFQaLxq1qxZRRVX3LRp0wrV3aRJkxLn0dX96ubmVqivCoUCo0ePLrK9Lu3Tw4cPo1evXnB2doZCocDWrVs1pgshMHXqVDg5OcHExARdunRBYmJiqcst72e+KpTU1ydPnuDTTz+Fl5cXzMzM4OzsjJCQENy6davEZVbkc1AVStuvgwcPLlR3165dS12uru1XAEV+dhUKBWbNmlXsMqvjfi3L98vjx48xevRo2NrawtzcHP369cOdO3dKXG5FP+PlxUBVCX799VeMHz8eERERiI+PR4sWLRAYGIi0tLQi2x89ehQDBw5EaGgoTp8+jd69e6N37964cOFCFVdefocOHcLo0aNx/Phx7N27F0+ePMFbb72FnJycEueztLREamqq9Lp27VoVVSxPs2bNNOr+888/i22ry/v1xIkTGv3cu3cvAKB///7FzqMr+zQnJwctWrTAokWLipz+3XffYcGCBVi8eDFiY2NhZmaGwMBAPH78uNhllvczX1VK6mtubi7i4+MxZcoUxMfHY/Pmzbh06RLefvvtUpdbns9BVSltvwJA165dNepeu3ZticvUxf0KQKOPqamp+Pnnn6FQKNCvX78Sl1vd9mtZvl/GjRuHHTt2YMOGDTh06BBu3bqFvn37lrjcinzGK0SQbL6+vmL06NHSsEqlEs7OziIqKqrI9gMGDBA9evTQGNe2bVsxYsSIl1rny5CWliYAiEOHDhXbZvny5cLKyqrqiqokERERokWLFmVur0/7dezYsaJBgwZCrVYXOV1X9ykAsWXLFmlYrVYLR0dHMWvWLGlcenq6UCqVYu3atcUup7yfeW14sa9FiYuLEwDEtWvXim1T3s+BNhTV10GDBomgoKByLUdf9mtQUJDo3LlziW10Yb+++P2Snp4ujIyMxIYNG6Q2CQkJAoA4duxYkcuo6Ge8IniESqb8/HycOnUKXbp0kcYZGBigS5cuOHbsWJHzHDt2TKM9AAQGBhbbvjrLyMgA8H8/AF2c7Oxs1KtXD66urggKCsJff/1VFeXJlpiYCGdnZ9SvXx/BwcFISUkptq2+7Nf8/Hz88ssvGDp0KBQKRbHtdHWfPu/q1au4ffu2xn6zsrJC27Zti91vFfnMV1cZGRlQKBSwtrYusV15PgfVSXR0NGrXrg0PDw+MHDkS9+/fL7atvuzXO3fuYOfOnQgNDS21bXXfry9+v5w6dQpPnjzR2EdNmjRB3bp1i91HFfmMVxQDlUz37t2DSqWCg4ODxngHBwfcvn27yHlu375drvbVlVqtRnh4ODp27IjmzZsX287DwwM///wztm3bhl9++QVqtRodOnTAjRs3qrDa8mvbti1WrFiBP/74Az/88AOuXr2KTp06ISsrq8j2+rJft27divT0dAwePLjYNrq6T19UsG/Ks98q8pmvjh4/foxPP/0UAwcOLPEHZcv7OaguunbtilWrVmH//v349ttvcejQIXTr1g0qlarI9vqyX1euXAkLC4tST4NV9/1a1PfL7du3YWxsXOg/AKV93xa0Kes8FVWjUpdGr5TRo0fjwoULpZ53b9++Pdq3by8Nd+jQAZ6envjxxx8xY8aMl11mhXXr1k167+3tjbZt26JevXpYv359mf73p6uWLVuGbt26wdnZudg2urpP6ZknT55gwIABEELghx9+KLGtrn4O3nvvPem9l5cXvL290aBBA0RHRyMgIECLlb1cP//8M4KDg0u9SaS679eyfr9UJzxCJZOdnR0MDQ0L3WVw584dODo6FjmPo6NjudpXR2FhYfjtt99w8OBB1KlTp1zzGhkZoVWrVrhy5cpLqu7lsLa2RuPGjYutWx/267Vr17Bv3z589NFH5ZpPV/dpwb4pz36ryGe+OikIU9euXcPevXtLPDpVlNI+B9VV/fr1YWdnV2zdur5fAeDIkSO4dOlSuT+/QPXar8V9vzg6OiI/Px/p6eka7Uv7vi1oU9Z5KoqBSiZjY2P4+Phg//790ji1Wo39+/dr/A/+ee3bt9doDwB79+4ttn11IoRAWFgYtmzZggMHDsDd3b3cy1CpVDh//jycnJxeQoUvT3Z2NpKSkoqtW5f3a4Hly5ejdu3a6NGjR7nm09V96u7uDkdHR439lpmZidjY2GL3W0U+89VFQZhKTEzEvn37YGtrW+5llPY5qK5u3LiB+/fvF1u3Lu/XAsuWLYOPjw9atGhR7nmrw34t7fvFx8cHRkZGGvvo0qVLSElJKXYfVeQzLqcDJNO6deuEUqkUK1asEBcvXhTDhw8X1tbW4vbt20IIIT788EPx2WefSe1jYmJEjRo1xOzZs0VCQoKIiIgQRkZG4vz589rqQpmNHDlSWFlZiejoaJGamiq9cnNzpTYv9jcyMlLs3r1bJCUliVOnTon33ntP1KxZU/z111/a6EKZTZgwQURHR4urV6+KmJgY0aVLF2FnZyfS0tKEEPq1X4V4dkdT3bp1xaefflpomi7v06ysLHH69Glx+vRpAUD85z//EadPn5bubJs5c6awtrYW27ZtE+fOnRNBQUHC3d1dPHr0SFpG586dxcKFC6Xh0j7z2lJSX/Pz88Xbb78t6tSpI86cOaPx+c3Ly5OW8WJfS/scaEtJfc3KyhITJ04Ux44dE1evXhX79u0T//rXv0SjRo3E48ePpWXow34tkJGRIUxNTcUPP/xQ5DJ0Yb+W5fvl3//+t6hbt644cOCAOHnypGjfvr1o3769xnI8PDzE5s2bpeGyfMYrAwNVJVm4cKGoW7euMDY2Fr6+vuL48ePSND8/PzFo0CCN9uvXrxeNGzcWxsbGolmzZmLnzp1VXHHFACjytXz5cqnNi/0NDw+Xto2Dg4Po3r27iI+Pr/riy+ndd98VTk5OwtjYWLi4uIh3331XXLlyRZquT/tVCCF2794tAIhLly4VmqbL+/TgwYNF/s0W9EetVospU6YIBwcHoVQqRUBAQKFtUK9ePREREaExrqTPvLaU1NerV68W+/k9ePCgtIwX+1ra50BbSuprbm6ueOutt4S9vb0wMjIS9erVE8OGDSsUjPRhvxb48ccfhYmJiUhPTy9yGbqwX8vy/fLo0SMxatQoUatWLWFqair69OkjUlNTCy3n+XnK8hmvDIr/v3IiIiIiqiBeQ0VEREQkEwMVERERkUwMVEREREQyMVARERERycRARURERCQTAxURERGRTAxURERERDIxUBER/X/Tpk2Dg4MDFAoFtm7dqu1yiEiHMFARUbndvn0bY8aMQf369aFUKuHq6opevXoV+i3DqlBZ4SchIQGRkZH48ccfkZqaim7duskvjoheGTW0XQAR6Zbk5GR07NgR1tbWmDVrFry8vPDkyRPs3r0bo0ePxt9//63tEiskKSkJABAUFASFQqHlasouPz8fxsbGGuOEEFCpVKhRg//EE1UVHqEionIZNWoUFAoF4uLi0K9fPzRu3BjNmjXD+PHjcfz4caldSkoKgoKCYG5uDktLSwwYMAB37tyRpg8ePBi9e/fWWHZ4eDj8/f2lYX9/f3z88cf45JNPYGNjA0dHR0ybNk2a7ubmBgDo06cPFAqFNFyU8+fPo3PnzjAxMYGtrS2GDx+O7OxsAM9O9fXq1QsAYGBgUGKgOnToEHx9faFUKuHk5ITPPvsMT58+laar1Wp89913aNiwIZRKJerWrYuvv/5amn7jxg0MHDgQNjY2MDMzQ+vWrREbG1uubRIWFobw8HDY2dkhMDAQ0dHRUCgU2LVrF3x8fKBUKvHnn39CrVYjKioK7u7uMDExQYsWLbBx40ZpWQXz7d+/H61bt4apqSk6dOiAS5cuadSwY8cOtGnTBjVr1oSdnR369OkjTcvLy8PEiRPh4uICMzMztG3bFtHR0cVuPyJ9xUBFRGX24MED/PHHHxg9ejTMzMwKTbe2tgbwLFQEBQXhwYMHOHToEPbu3Yt//vkH7777brnXuXLlSpiZmSE2Nhbfffcdpk+fjr179wIATpw4AQBYvnw5UlNTpeEX5eTkIDAwELVq1cKJEyewYcMG7Nu3D2FhYQCAiRMnYvny5QCA1NRUpKamFrmcmzdvonv37mjTpg3Onj2LH374AcuWLcNXX30ltZk8eTJmzpyJKVOm4OLFi1izZg0cHBwAANnZ2fDz88PNmzexfft2nD17Fp988gnUanW5t4mxsTFiYmKwePFiafxnn32GmTNnIiEhAd7e3oiKisKqVauwePFi/PXXXxg3bhw++OADHDp0SGN5X3zxBebMmYOTJ0+iRo0aGDp0qDRt586d6NOnD7p3747Tp09j//798PX1laaHhYXh2LFjWLduHc6dO4f+/fuja9euSExMLFefiHRepf/cMhHprdjYWAFAbN68ucR2e/bsEYaGhiIlJUUa99dffwkAIi4uTgghxKBBg0RQUJDGfGPHjhV+fn7SsJ+fn3jttdc02rRp00Z8+umn0jAAsWXLlhLrWbJkiahVq5bIzs6Wxu3cuVMYGBiI27dvCyGE2LJliyjtn8TPP/9ceHh4CLVaLY1btGiRMDc3FyqVSmRmZgqlUimWLl1a5Pw//vijsLCwEPfv3y9yelm3SatWrTTaHDx4UAAQW7dulcY9fvxYmJqaiqNHj2q0DQ0NFQMHDtSYb9++fdL0nTt3CgDi0aNHQggh2rdvL4KDg4us99q1a8LQ0FDcvHlTY3xAQICYPHlykfMQ6SueYCeiMhNClKldQkICXF1d4erqKo1r2rQprK2tkZCQgDZt2pR5nd7e3hrDTk5OSEtLK/P8BfW0aNFC46hax44doVarcenSJekIUlmW0759e41Tgh07dkR2djZu3LiB27dvIy8vDwEBAUXOf+bMGbRq1Qo2Njblqv9FPj4+RY5v3bq19P7KlSvIzc3Fm2++qdEmPz8frVq10hj3/DZ2cnICAKSlpaFu3bo4c+YMhg0bVuT6zp8/D5VKhcaNG2uMz8vLg62tbdk7RKQHGKiIqMwaNWoEhUJRKReeGxgYFApoT548KdTOyMhIY1ihUJT7FFlVMTExkTW9rNukqNOtL44vuD5s586dcHFx0WinVCo1hp/fxgVhsWAbl1RzdnY2DA0NcerUKRgaGmpMMzc3L3Y+In3Ea6iIqMxsbGwQGBiIRYsWIScnp9D09PR0AICnpyeuX7+O69evS9MuXryI9PR0NG3aFABgb29f6FqlM2fOlLsmIyMjqFSqEtt4enri7NmzGjXHxMTAwMAAHh4eZV6Xp6cnjh07phF6YmJiYGFhgTp16qBRo0YwMTEp9vER3t7eOHPmDB48eFDk9MraJsCzI4JKpRIpKSlo2LChxuv5I4el8fb2LrY/rVq1gkqlQlpaWqF1ODo6VqhuIl3FQEVE5bJo0SKoVCr4+vpi06ZNSExMREJCAhYsWID27dsDALp06QIvLy8EBwcjPj4ecXFxCAkJgZ+fn3RaqnPnzjh58iRWrVqFxMRERERE4MKFC+Wux83NDfv378ft27fx8OHDItsEBwejZs2aGDRoEC5cuICDBw9izJgx+PDDD8t8ug94dofj9evXMWbMGPz999/Ytm0bIiIiMH78eBgYGKBmzZr49NNP8cknn2DVqlVISkrC8ePHsWzZMgDAwIED4ejoiN69eyMmJgb//PMPNm3ahGPHjlXqNgEACwsLTJw4EePGjcPKlSuRlJSE+Ph4LFy4ECtXrizzciIiIrB27VpEREQgISEB58+fx7fffgsAaNy4MYKDgxESEoLNmzfj6tWriIuLQ1RUFHbu3Fmhuol0lnYv4SIiXXTr1i0xevRoUa9ePWFsbCxcXFzE22+/LQ4ePCi1uXbtmnj77beFmZmZsLCwEP3795cuAC8wdepU4eDgIKysrMS4ceNEWFhYoQuwx44dqzFPUFCQGDRokDS8fft20bBhQ1GjRg1Rr169Yms+d+6ceOONN0TNmjWFjY2NGDZsmMjKypKml+WidCGEiI6OFm3atBHGxsbC0dFRfPrpp+LJkyfSdJVKJb766itRr149YWRkJOrWrSu++eYbaXpycrLo16+fsLS0FKampqJ169YiNjZW1jYpuLj84cOHGuPVarWYN2+e8PDwEEZGRsLe3l4EBgaKQ4cOFTvf6dOnBQBx9epVadymTZtEy5YthbGxsbCzsxN9+/aVpuXn54upU6cKNzc3YWRkJJycnESfPn3EuXPnSt2WRPpEIUQZrzIlIiIioiLxlB8RERGRTAxURERERDIxUBERERHJxEBFREREJBMDFREREZFMDFREREREMjFQEREREcnEQEVEREQkEwMVERERkUwMVEREREQyMVARERERycRARURERCTT/wObLCpgH4G5MgAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer\n", "\n", "# Instatiate, fit and transform - Count vectorization\n", "cv = CountVectorizer(stop_words='english')\n", "count_vectorized = cv.fit_transform([text])\n", "\n", "# Recreate the dataframe\n", "count_df = pd.DataFrame(count_vectorized.todense(), columns=cv.get_feature_names_out())\n", "\n", "# Visualize again\n", "count_df.T[0].sort_values().tail(10).plot(kind='barh')\n", "plt.xlabel('Count of occurrence')\n", "plt.ylabel('Token')\n", "plt.title('Most Frequently Occuring Tokens')\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": { "id": "gI8wviJdfdnD" }, "source": [ "This seems to be more representative, given what we know about the topic of the original text. Without doing any machine learning, we have done some basic text analytics (content analysis) given we now have structured data!\n", "\n", "Here we have performed the basics of the preprocessing steps for text on a single document. In practice, this would be done over a very large corpus of many documents, and the document-term matrix produced from vectorization can grown quite large." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "----\n", "\n", "\n", " \n", " \n", " \n", " \n", "\n", "

Copyright NLP from scratch, 2024.

" ] } ], "metadata": { "colab": { "provenance": [ { "file_id": "https://github.com/mylesmharrison/nlp4free/blob/master/notebooks/NLP4Free_Part2_DataAcquisitionandPreprocessing.ipynb", "timestamp": 1687980773112 } ] }, "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.6" } }, "nbformat": 4, "nbformat_minor": 4 }